The BUILD initiative is part of the Diversity Program Consortium, which the National Institutes of Health funded to increase diversity in biomedical research. This chapter aims to identify implications for the field from the multisite evaluation of BUILD initiative programs by reviewing the work undertaken by the authors of the other chapters in this issue. Given the complexities involved in multisite evaluations, innovative approaches and methods were used to balance the needs of each site with the overall objectives of the broader initiative. These approaches included a flexible orientation to the evaluation, mixed-methods designs that prioritized understanding the context before measuring it, and innovative analytic techniques  to recognize the uniqueness of each site while providing insights about their cumulative impact. The BUILD initiative evaluation also offered many other valuable lessons about engaging stakeholders, focusing on use, and responding to changing priorities over time. Large scale multisite evaluations are difficult yet worthwhile because they force us to consider local contextual factors while maintaining a broader perspective on the overall initiative. This challenge, if approached effectively, has many potential benefits, including a better understanding of how policies can change trends and how they are translated to respond to the needs of different communities. Previous multisite evaluations have tended to focus on differing elements, including the methodological approaches involved in the process , the rolNew Directions for Evaluation issue to this growing knowledge base is evident. The preceding chapters have explored the complex evaluation of the Building Infrastructure Leading to Diversity (BUILD) initiative, funded by the National Institutes of Health as part of the Diversity Program Consortium (DPC) to increase the diversity of students who pursue biomedical research careers. The evaluation is overseen by the initiative’s Coordination and Evaluation Center (CEC), housed at the University of California, Los Angeles.The contribution of this As I reviewed the work of my colleagues, I was struck by the myriad methodological considerations, analytic techniques, and responsiveness to context and stakeholders that emerged across this evaluation endeavor. Together, these chapters provide one of the few examples of large-scale multisite evaluations that have been documented and presented from varying perspectives, including the technical, cultural, and theoretical; they also provide new insights that can inform our practice. The hope is that the lessons learned from this project will inform future evaluation efforts across sectors and contexts.utos, where u = units (people); t = treatment (intervention); o = observation (measures); and s = setting (context of intervention).1To help frame the insights gained in this chapter I utilize Lee Cronbach’s ideas on generalizability. In his approach to evaluation, time as fifth element to show how multiyear programs change and evolve as they develop. Overall, this way of representing the context may help offer insights into some of the broad lessons learned from the efforts of the DPC. My goal is to describe its implications to the evaluation field.At its core, this approach attempts to represent how an initiative has responded to the uniqueness of its environment by comparing and contrasting various small evaluation studies of the program. This process can lead to insights about the environmental factors that have enhanced or hindered success within differing contexts. It is also worth noting that 2. Understanding whether this broad goal has been reached requires knowledge of who was served as part of the program and the types of training needed to support their potential for success.The overall purpose of BUILD was described in Chapter 1. Specifically, the initiative has the ultimate goal of “[enhancing] the diversity of well-trained biomedical research scientists who can successfully compete for NIH research funding and/or otherwise contribute to the NIH-funded workforce”u in the utos model, as these scholars focused on the individuals the program recruited, retained, and supported. From an evaluation perspective, the efforts described in Chapter 5 by Maccalla et al. represented a potentially effective way of understanding who the participants were in terms of their position (student or faculty), relevant information about their role in the program , and a host of other background characteristics. From an innovation standpoint, the systematic approach described in Chapter 5 allowed the evaluation to reveal how unique individual characteristics across various sites interacted with the contexts within which they were working, while also providing a broader perspective on the achieved outcomes. This ability to view the forest and trees offered information on who participated and how long they remained part of the program and ultimately helped identify interactions between participants and the outcomes that emerged from their program engagement. This also helped answer questions about whom the program worked for and possibly about why it worked for them. Part of that answer also relied on how the “program” was defined at each of the sites.The diversity of students and faculty across sites were the t in the utos framework. As acknowledged in Chapter 1 (and in other chapters), this initiative was implemented heterogeneously across 10 sites, and the programing changed over time. As such, there were differences between sites and within sites over time. The difficulty of disentangling the changing program characteristics across sites and time and then identifying the program’s relationship to various measured outcomes would pose a great challenge to an evaluator. These measures can be the observations, or the o in the utos framework. Given this challenge, I believe that this evaluation did an excellent job of developing various qualitative and quantitative approaches to capture programmatic activities and outcomes and speak to the potential impacts of these efforts within and across sites.How the program was defined at each site represents the treatment or t in utos), or help the evaluation gain a more accurate understanding of the program outcomes and how to optimally measure them .For example, if we think about the various evaluation activities that the CEC conducted, we can detect a pattern that mimics an exploratory sequential mixed-methods design . In thiswhy there was heterogeneous implementation, and how these varying program models connected to the outcomes. I am unsure how the information gathered from the case study informed other pieces of this evaluation, as this was not explicitly stated in the other chapters. Given the structure of the remaining evaluation efforts, the findings from these case studies may have formally or informally guided some of the subsequent quantitative processes present throughout the evaluation.After reviewing and reflecting on the various chapters as a whole, the mixed-methods approach appeared most likely occurred during the case study portion of the evaluation  and when identifying Hallmarks of Success . In the case study portion, the authors noted that “Program evaluation efforts that utilize case study design can provide a rich, in-depth understanding of program effectiveness that can either supplement quantitative evaluation measures in an explanatory, mixed methods evaluation to increase understanding of program outcomes, or stand alone in presenting multi-faceted dimensions of program implementation” . This initial qualitative study looked at factors that may have influenced how each site designed an initiative that best suited its specific needs, history, resources, capacity, students, and faculty. This qualitative study also offered insights into t in utos. Even though this phase was iterative in nature, it aided the development of a quantitative tracking tool that operationalized many of the qualitatively described program components that emerged from the initial exploratory phase. Quantitative measures included enrollment in new courses developed across the sites and participation in career advancement activities, such as career advising, learning communities, and undergraduate research experiences. The quantitative tracking also extended to faculty involvement in program activities, including research training and support and conference participation.The exploratory mixed-methods approach was also described by Maccalla et al. in Chapter 5, where an initial qualitative process was implemented to define what a “program” is. This initial qualitative exploratory phase included document review, member checking, and thematic coding to understand the various program components and how they were operationalized across sites. This process also offered insights about the treatment, or the This tracking system allowed the evaluation to compare program implementation by creating categories of activities that were broad enough to be applicable across the 10 sites yet detailed enough to reflect the unique focus of each site. This approach helped provide the evaluation with a perspective that balanced the forest and trees. Even though the mixed-methods approach was not explicitly stated in the chapter, the approach as described fits the definition of an exploratory sequential mixed-methods design , with tho in utos). This has some very interesting implications for the field of evaluation, since the approach leveraged meta-analysis to provide an understanding of what occurred at each site and what happened across sites. As the authors noted, meta-analysis has historically been used to aggregate findings from hundreds or thousands of studies. From a conceptual perspective, a meta-analytic approach takes differing outcome measures from across studies and standardizes the outcomes by calculating their effect sizes, and then uses that information to calculate the absolute effect size to determine the overall impact of an intervention  used the meta-analytic approach in an innovative way by leveraging its main advantage—the ability to standardize findings from across 10 program sites—and accounting for different sample characteristics and different implementation plans. They calculated these effect sizes for each program site and then used them in the broader meta-analysis to provide an aggregated absolute effect of the entire initiative. This is a relevant contribution to evaluation practice because it permits different sites to use their own conceptualizations of quantitative outcomes and activities. These outcomes and activities may best represent the unique goals they wish to influence and the activities that would optimally serve their students while still allowing the evaluation to combine findings at a broader level to determine overall initiative impact. This effort also builds on previous work conducted by utos framework is examining the setting, or the s, and the influence of contextual characteristics, such as culture, history, and the surrounding environment, on programs and their outcomes. This element was not described in great detail within the preceding chapters but was captured in the overall evaluation theories that helped guide this effort. As mentioned in Chapter 6 by Christie and Wright, the primary theoretical framework that was adopted was utilization—specifically, utilization-focused evaluation  to gain a broader perspective on the initiative’s impact. This balancing act between standardization and flexibility is at the core of any multisite evaluation, and it was a tangible struggle in this specific effort. However, I believe that the chapter authors overcame many of these challenges and were able to respond to the uniqueness of each site by thoughtfully acknowledging the individuals, the activities, the outcomes, and the settings while also identifying the overall impacts of this large-scale effort. Many lessons have been learned from this process, and I hope that the evaluation community has gained a better perspective from this valuable work.


Introduction: Otolaryngology remains one of the most competitive specialties to match into during the residency application process. Medical students often apply to many programs to increase their chances of matching into a residency program and rely on residency websites to gather information about the programs they apply to. The purpose of this study was to determine the comprehensiveness of the information on otolaryngology residency program websites.Methods: One hundred twenty-two publicly available otolaryngology residency program websites were evaluated for the presence of 47 criteria. Size, geographic location, and affiliation with a Top 50 ranked hospital for ear, nose, and throat care, according to the US News and World Report, were determined for each program. Frequencies were calculated for each of the different residency website criteria and non-parametric comparisons were used to analyze the relationship between the location, size, and ranking of each program, and the comprehensiveness of the program website.Results: An average of 19.1 items (SD: 6.6 items) out of the 47 searches were present on the otolaryngology residency program websites. More than 75% of the websites contained the following program features: facility descriptions, descriptions of didactics, and research requirements. A total of 89.3% of the websites had a current resident list, 87.7% of these websites had photos of their residents, and 86.9% had a program contact email. Otolaryngology residency programs affiliated with a Top ENT hospital had a higher average number of criteria satisfied (21.6 criteria) compared to those that were not affiliated (17.9 criteria).Conclusion: The inclusion of research selection criteria, call schedule/requirements, average Step 2 scores of matched residents, and social aspects of residency could improve otolaryngology residency applicant satisfaction with residency program websites. Updating otolaryngology residency websites will assist prospective applicants as they apply to a wide variety of residency programs. Almost every year, there are more applicants than the actual number of residency spots ,2. Due tApplying to more programs with the belief that this will lead to more interviews and a better chance to match has been labeled the “shotgun” approach . HoweverIn congruence with their desire to match in a highly competitive field, otolaryngology applicants will apply to a large number of programs, often due to an inability to definitively differentiate between programs. Many applicants utilize the internet when trying to compare residency programs. This implies that residency program websites are a key component in helping applicants decide where to apply. Therefore, the content of otolaryngology residency program websites is of great interest to applicants, with the potential to help them decide which programs may be their best fit. This study seeks to determine the current comprehensiveness of the information on otolaryngology residency program websites.One hundred twenty-two publicly available otolaryngology residency program websites were evaluated for the presence of information that would be relevant to prospective otolaryngology residency program applicants in July 2021. The total list of 125 programs was accessed through the American Medical Association FREIDA (Fellowship and Residency Electronic Interactive Database). Two programs did not have a website linked to FREIDA and one program had no information on its webpage. After identifying the programs, four of the authors  searched for a total of 47 different criteria on the websites. Criteria needed to be present directly on the page or accessible through a listed link or search tab on the program website. Links that were not directly on the program website or able to be found via the search engine were not accessed. These 47 criteria were determined based on the results of a prior study conducted by DiNardo et al. [Both the size and the geographic location of the otolaryngology residency programs were classified. Program size refers to the number of otolaryngology residents accepted into each program per year. A program was classified as large if it accepted three or more residents per year and small if it accepted two or less. Programs that alternated between two and three residents per year were classified as small based on the mean number of residents per year being below three. The location of each program was categorized as Northeast, Midwest, West, and South based on the U.S. Census Bureau classification . Lastly,Frequencies were calculated for each of the different residency website criteria. Non-parametric comparisons were also utilized in order to analyze the relationship between the location, size, and ranking of each program, and the comprehensiveness of the program website. The methods of this study were created based on two prior studies that evaluated residency program websites ,11.Information on the residency program region, size, and whether the program was affiliated with a top 50 otolaryngology hospital can be found in Table Out of the 47 factors searched for on the websites, programs contained an average of 19.1 items (SD: 6.6 items). Table Otolaryngology residency program websites assist applicants in determining if a residency program is a good “fit” for them. These websites gained importance during the virtual interviews of the 2020-2021 application season, and continue to be important as otolaryngology programs offer virtual interviews. This study sought to evaluate the comprehensiveness of otolaryngology residency program websites, as residency programs continue to update their websites to support the virtual nature of residency program exploration during the COVID-19 pandemic.This study expands upon an analysis of otolaryngology residency program websites that was conducted in 2014 . Svider The criteria evaluated in this study were determined based on a survey study conducted by DiNardo et al. that surveyed 101 fourth-year medical students applying to otolaryngology, re-applicants applying to otolaryngology, and current first-year otolaryngology residents . A totalThe United States Medical Licensing Examination (USMLE) Step 1 and Step 2 score minimum requirements were only listed on less than 10% of the websites and few websites contained average Step 1 and Step 2 scores of current residents . The lacSvider et al. and Tong et al. did not evaluate any website components associated with social aspects of the prospective resident’s life, with the exception of a video highlighting the resident's life and photos of the resident's activities in Tong et al.’s study ,12. Our Despite the increase in advocacy for diversity in medicine that has occurred in the last decade, only 31 (25.4%) of the program websites contained a statement that reflected their stance on the diversity of their program or their commitment to diversity. These statistics corroborate the findings of Driesen et al., who sought to determine the presence of diversity elements on general surgery program websites . One of While the comprehensiveness of otolaryngology residency program websites has increased since the 2014 study by Svider et al., many websites still lack components that candidates may deem useful when deciding where to apply and creating their rank list. A survey of prospective otolaryngology residency applicants in the 2020-2021 match found that 53 (62.4%) applicants did not believe they would gather sufficient information to make informed decisions about their own rank list [LimitationsSince each otolaryngology program website was very different, the aesthetic qualities and ease of use of the website were not quantified. Additionally, this study only looked at criteria present directly on the page, or accessible through a listed link or search tab on the program website page as of July 2021. Thus, there may have been criteria, such as salary and benefits, listed elsewhere on the hospital website that was not found and tallied. The program websites could have been updated between the time of analysis and the publication of this study. Overall, a major limitation was also the subjectivity in determining which criteria to include. However, the authors utilized DiNardo et al.’s study and attempted to be comprehensive in the criteria in order to diminish the effect of this limitation.With an increased focus on virtual interviews during the COVID-19 pandemic, otolaryngology residency programs should update their websites to include comprehensive criteria in order to attract prospective applicants. The presence of research selection criteria, call schedule/requirements, average Step 2 scores of matched residents, and social aspects of residency could improve otolaryngology residency applicant satisfaction regarding residency program websites. Updating these websites will assist prospective applicants in making informed decisions about where they wish to apply and thus, reduce the high costs of the residency application season.


Engagement is key to interventions that achieve successful behavior change and improvements in health. There is limited literature on the application of predictive machine learning (ML) models to data from commercially available weight loss programs to predict disengagement. Such data could help participants achieve their goals.This study aimed to use explainable ML to predict the risk of member disengagement week by week over 12 weeks on a commercially available web-based weight loss program.Data were available from 59,686 adults who participated in the weight loss program between October 2014 and September 2019. Data included year of birth, sex, height, weight, motivation to join the program, use statistics , program type, and weight loss. Random forest, extreme gradient boosting, and logistic regression with L1 regularization models were developed and validated using a 10-fold cross-validation approach. In addition, temporal validation was performed on a test cohort of 16,947 members who participated in the program between April 2018 and September 2019, and the remaining data were used for model development. Shapley values were used to identify globally relevant features and explain individual predictions.The average age of the participants was 49.60 (SD 12.54) years, the average starting BMI was 32.43 (SD 6.19), and 81.46%  of the participants were female. The class distributions (active and inactive members) changed from 39,369 and 9235 in week 2 to 31,602 and 17,002 in week 12, respectively. With 10-fold-cross-validation, extreme gradient boosting models had the best predictive performance, which ranged from 0.85 (95% CI 0.84-0.85) to 0.93 (95% CI 0.93-0.93) for area under the receiver operating characteristic curve and from 0.57 (95% CI 0.56-0.58) to 0.95 (95% CI 0.95-0.96) for area under the precision-recall curve (across 12 weeks of the program). They also presented a good calibration. Results obtained with temporal validation ranged from 0.51 to 0.95 for area under a precision-recall curve and 0.84 to 0.93 for area under the receiver operating characteristic curve across the 12 weeks. There was a considerable improvement in area under a precision-recall curve of 20% in week 3 of the program. On the basis of the computed Shapley values, the most important features for predicting disengagement in the following week were those related to the total activity on the platform and entering a weight in the previous weeks.This study showed the potential of applying ML predictive algorithms to help predict and understand participants’ disengagement with a web-based weight loss program. Given the association between engagement and health outcomes, these findings can prove valuable in providing better support to individuals to enhance their engagement and potentially achieve greater weight loss. Chronic conditions, such as cardiovascular disease and diabetes, are the leading cause of poor health and death in Australia  and in oThere has been an exponential growth in digital behavioral interventions to prevent chronic diseases and promote health . As a reEngagement is thought to be critical to interventions achieving successful behavior change and health improvements. Engagement is defined as “the extent to which, and how, individuals participate in an intervention” . EngagemApplying machine learning (ML) to digital health and weight loss programs could potentially improve the experience for participants by providing more tailored content, in turn increasing their engagement in the intervention and improving the health outcomes achieved by participants. ML has been applied in various ways to address a range of health conditions, such as the detection of diseases such as dementia in individuals , diagnosThis presented an opportunity for this study to address several limitations in the current body of literature, including the application of ML to a real-world weight loss intervention that has weight and engagement data collected at regular intervals from a larger sample of participants compared with previously published research. This study used data from a web-based commercial weight loss program to predict a user’s risk of disengagement during a 12-week program. The ability to predict disengagement results before the end of a program could be particularly useful to identify program participants at risk of premature departure and to inform when additional support could be directed toward at-risk individuals, possibly increasing the overall success of the program and outcomes for the participants. Therefore, this study aimed to (1) investigate the ability of state-of-the-art ML algorithms to predict member disengagement week by week, which would allow better support of participants and possibly better weight loss outcomes, and (2) explain the output of the developed predictive models.The participants were adults (aged ≥18 years) who joined the Commonwealth Scientific and Industrial Research Organisation (CSIRO) Total Wellbeing Diet Online program between October 2014 and September 2019. CSIRO Total Wellbeing Diet Online [In total, there were 59,686 unique participant IDs over the 5-year study period. This analysis included participants who selected the 12-week weight loss program with or without dietitian assistance. Some participants were members for longer than the initial 12-week program; however, this analysis used data from the first 12 weeks of participation only. After the exclusion of participants with invalid or missing feature entries and those selecting a program type shorter than 12 weeks, the resulting cohort for analysis was 81.43%  of all unique participant IDs. Details of the exclusion process are shown in The primary outcome of this study was the inactivity (disengagement) on the platform. Activity related to the use of 7 features, namely, entries into the food diary, views of the menu plans, views of exercise plans, views of program content information, forum visits, searches of the food database, and weight entries. The outcome of interest was inactivity (no engagement) in the following week, which was defined as the absence of all 7 activity counts within the following week. This resulted in 11 outcome metrics, each corresponding to 1 week of the program, starting from the second week. The absence of any activity on the platform was of interest as it was possibly an indication of imminent withdrawal from the program.The predictors used in this study were divided into 2 groups: static and dynamic features. The static features included demographic predictors, that is, age; sex; starting BMI; Socioeconomic Indexes for Areas, Index of Relative Socioeconomic Disadvantage decile, with decile 1 representing areas with the most disadvantage or least advantage and decile 10 representing areas with the least disadvantage or most advantage; and predictors anticipated to reflect the motivation for participation, that is, the number of days between setup and nominated start date and the reason  for weight loss. Each reason for weight loss was represented as a categorical variable, with 1 representing its presence and 0 representing its absence. To explore the impact of the opt-in dietitian support, a categorical feature was created and set to 1 for those who chose the program with support from a dietitian and to 0 for those who did not. The dynamic group of features included variables that were subject to change weekly. These included counts of individual activities , the sum of individual activities denoted as total activity, and accumulative weight loss. The incremental weight change between 2 adjacent weeks of the program was calculated and included as a predictor. For each predicted outcome metric, starting from week 2, all static and dynamic features available up to that week were considered as predictors. For example, for predicting activity in week 4, all dynamic features available for weeks 1, 2, and 3 were considered alongside the static feature group. The list of all features considered in this study is provided in Table S1 in In this study, we considered logistic regression with L1 regularization (L1), random forest (RF), and extreme gradient boosting (XGB) as candidate models. L1, popularly known as the least absolute shrinkage and selection operator, was chosen because of its established efficacy in solving prediction problems and ability to perform feature selection to increase model parsimony . The latTo capture variability, model validation was conducted with nested 10-fold cross-validation 10FCV; . Nested We did not perform hyperparameter optimization on the training data but instead used the model structure (model hyperparameters) as the best parameters in the 10FCV process.F1-score metrics, as they are also suitable for assessing model performance where the data are imbalanced. To ensure the reliability of the predictions, calibration curves were considered during the selection process for the final models. Calibration curves were generated by grouping the predictions into quantiles (we used 10), each represented as a point. The x-axis of the calibration curve shows the proportion of true outcomes, and the y-axis shows the mean predicted probability. The better the calibrated model, that is, the more reliable the forecast, the closer the points are along the main diagonal. The equations for computing the evaluation metrics are provided in To evaluate the model performance, we used a set of performance indicators. The area under the receiver operating characteristic curve (AUC-ROC) was used to measure model discrimination. Although AUC-ROC is the gold standard for quantification of the discriminative power of predictive models developed for clinical applications, it is regarded as a misleading performance indicator for imbalanced data sets ,16. Thert test. To compare the performance means of the models, one-way ANOVA tests were applied.Reported performance means and 95% CIs of 10FCV obtained on test partitions were calculated by averaging and computing the CIs using the Student 2-tailed explainer) to explain the relationship between the inputs and predicted outputs. To analyze the relevance of predictors in predicting the program participants at risk of disengagement, we created the explainer based on Shapley values introduced in the study by Lundberg et al [Unlike regression models, which have an elegant expression that allows direct insight into the model, tree-based ML models require an additional element  package [scipy.stats.All models were trained in Python (version 3.6) software. We used the  package . The anaEthics approval to conduct this research was received from the CSIRO Health and Medical Human Research Ethics Committee . A more detailed outline of the weight loss program and data collection process has been published previously .A demographic summary of the cohort is provided in The class distribution for each outcome metric in the training and test partitions is shown in The total number of features used for modeling included 10 static features and 10 dynamic features , AUC-PRC, precision, recall, F1-score metrics. Overall, all performance metrics improved faster in the first half of the program, after which they continued to improve, albeit at a slower rate.The results showed that predicting activity in week 2 was challenging, and all metrics improved remarkably in week 3. The AUC-ROC performance of all 3 models improved in week 3 from 81% to 88% for L1, 84% to 88% for RF, and 84% to 89% for XGB; for all candidate models, the AUC-PRC performance in week 3 improved by at least 20%. The same performance improvement was observed for the precision, recall, and P values are provided in The computed As a part of the model development process, the calibrations of the final models for L1, RF, and XGB were evaluated. The XGB models had the best calibration across all outcome metrics. Except for the high probabilities in week 2, which were overforecasted, the models calibrated well across the whole probability range for each class and, as such, were considered reliable. To analyze the Shapley value–based feature contribution, we constructed explainers using the training data obtained with the temporal splitting and the XGB models. Summary plots of Shapley values obtained from test data for models predicting activity in weeks 2, 6, and 12 are shown in In week 2, the features that contributed the most to predicting the disengagement were entering the week-1 weight, total activity on the platform during the previous week, having the short time between setting up and starting the program, viewing the menu plans, and using the food diary in week 1. As the week progressed, continued activity on the platform and weighing in remained the most important features. Interestingly, in the middle parts of the program , weight loss (difference between weeks 1 and 2 and weeks 2 and 3) as well as total activity in the previous weeks were important features for predicting disengagement. Further examination of the summary plots showed that higher weight loss pushed the prediction toward engagement (class 0), whereas lower weight loss pushed the prediction toward disengagement (class 1). Participants with fewer days between the application setup and when they were nominated to start the program also pushed the prediction toward engagement.The purpose of this study was to determine how well state-of-the-art ML models predict week-by-week disengagement with a web-based, commercial weight loss program and explain the output of the developed predictive models. To the best of our knowledge, this is the first study to use ML algorithms to predict weekly disengagement in a web-based, real-world weight loss program. Three modeling approaches were tested: the regression-based model L1 and 2 tree-based ML approaches—the RF and XGB models. The models were run to predict disengagement each week during the 12-week weight loss program using all available data from previous weeks. Data about individuals when they signed up at the start of the program were available, as well as individuals’ weekly weight loss and weekly use of 7 features that form part of the web-based platform. Across all predictive models, the results indicated that having a weight recorded in the system and the total activity on the platform from the previous week or weeks were the most important explanatory variables. The performance of the XGB and RF models were similar for the first 3 weeks, although the XGB models performed best overall, with the AUC and AUC-PRC values for the XGB models ranging from 0.85 (95% CI 0.84-0.85) to 0.93 (95% CI 0.93-0.93) and from 0.57 (95% CI 0.56-0.58) to 0.95 (95% CI 0.95-0.96), respectively, across the program. The L1 model systematically had the worst discrimination. The better performance of tree-based methods could be explained by the fact that they inherently include nonlinear effects and interactions, whereas L1 only considers those explicitly included. Good calibration of the final XGB models across all probabilities provides an opportunity to use calibration curves to translate probabilistic output values into susceptibility groups, which can be additionally investigated and used in the future to create additional support for subgroups in greatest need. Depending on the program’s capabilities, this additional support could be delivered via digital media, such as email, SMS text messaging, and app notifications, or with a mix of more traditional media, such as telephone consultation with a health professional. Although the output of the predictive models suggested that having access to additional support from a dietitian via 3 phone or video call consultations was not highly relevant to predicting disengagement, others have suggested that combining human and digital support could help improve engagement  or reengThis study has shown the potential of applying ML algorithms to help predict disengagement with a weight loss program; however, the relationships between the predictors and disengagement should not be considered casual. One key strength of this study was the large sample size. A review of the applications of ML in health reported an average sample size of <100 people ; howeverMany studies have shown a positive association between engagement in digital health programs and better health outcomes for participants ,12,28,29Psychological literature suggests that past behavior is one of the best predictors of future behavior and that the strength of this relationship relates to the period . The resThe use of ML models in real-world health interventions is not common, and very few studies have applied ML in real-world weight loss programs. A review by Triantafyllidis and Tsanas  found 8 Future research is also required to explore the inclusion of other data such as other lifestyle behaviors and personality characteristics, reported when signing up or during the first few weeks of the program. This could improve the identification of the behavioral patterns of those members prone to disengagement in the early weeks of the program. However, the results imply that the information acquired during week 2 remarkably improved the identification of disengagement. The algorithm used in this study could be improved and generalized to other cohorts with a similar case mix and diet program using similar data. Eventually, the integration of the proposed algorithm into the existing algorithm of the platform would require programing that would activate an appropriate model depending on the week of the program. To overcome this, a dynamic model might be worthy of consideration in future research.Given the importance of engagement in achieving behavior change and realizing health improvements, there have been calls for greater emphasis on engagement in digital health research. This study showed the potential of applying ML predictive algorithms to help predict and understand predicted participants’ disengagement with a web-based weight loss program. The most important features for predicting week-by-week disengagement were related to individuals’ total activity on the platform and entering their weight into the platform in the weeks prior. The models developed for this study have evolved in an attempt to fill the gap in understanding the informative relationships between the predictors and disengagement within a web-based weight loss program. This knowledge may help improve retention and better support individuals in engaging in a timely and more effective way. Future research will focus on the evaluation of developed algorithms in a trial, the influence of collinearity between some features on the explanations, and exploiting the potential benefit of including other information available before the program starts, which could possibly improve the prediction of disengagement.


Substantial advances in the field have revealed novel ChREBP functions. Indeed, due to its actions in different tissues, ChREBP modulates the inter-organ communication through secretion of peptides and lipid factors, ensuring metabolic homeostasis. Dysregulation of these orchestrated interactions is associated with development of metabolic diseases such as type 2 diabetes (T2D) and non-alcoholic fatty liver disease (NAFLD). Here, we recapitulate the current knowledge about ChREBP-mediated inter-organ crosstalk through secreted factors and its physiological implications. As the liver is considered a crucial endocrine organ, we will focus in this review on the role of ChREBP-regulated hepatokines. Lastly, we will discuss the involvement of ChREBP in the progression of metabolic pathologies, as well as how the impairment of ChREBP-dependent signaling factors contributes to the onset of such diseases.Carbohydrate response element binding protein (ChREBP) is a glucose responsive transcription factor recognized by its critical role in the transcriptional control of glycolysis and  ChREBP expression is mostly abundant in liver and white/brown adipose tissues (Nrf2)), as well as pro-oxidative/apoptotic pathways (thioredoxin interacting protein (Txnip)) (Glut2/5)), ketohexokinase (Khk), aldolase B (AldoB), triose phosphate isomerase (TrioK) and lactate dehydrogenase (Ldh) in the small intestine (Pparγ))  has emerged as an essential metabolic coordinator. ChREBP is the transcription factor mediating the response to carbohydrates. In the presence of glucose metabolites, glucose-response activation conserved element (GRACE) domain is stimulated, inducing ChREBP translocation to the nucleus. The reversal mechanism occurs through the activation of low-glucose inhibitory domain (LID), which in turn inhibits GRACE activity under low glucose concentrations. Both domains are enclosed in the Mondo conserved region (MCR), a highly evolutionarily conserved domain located in N-terminus of the Mondo family members . Of note1 (Acc1)  – more d tissues  but it i tissues , small i tissues  and to a tissues , adrenal tissues  and brai tissues . ChREBP (Txnip)) , 12 and ntestine . In whit(Pparγ))  and the (Pparγ)) . Finally(Pparγ))  and hepa(Pparγ)) , both ofrosstalk , 17, the1.2de novo lipogenesis . Finally, palmitate undergoes the desaturation and elongation reactions respectively by stearoyl-CoA desaturases (SCDs) and fatty acyl-CoA elongase (ELOVL) family members triggering fatty acid production, including stearic acid, palmitoleic acid and oleic acid , also involved in VLDL metabolism  to reach peripheral organs , 24. Thitrations , 27. Nevtabolism , reinfor1.3de novo lipogenesis in WAT, and impairs hepatic insulin sensitivity, emphasizing the role of ChREBPβ in the communication between WAT and liver , systemic insulin action and energy balance  and a co-receptor β-Klotho. While its expression is more abundant in the liver, s system . FGF21 is system . In micesitivity , 37. Simsitivity , indicatsitivity .Fgf21 through ChREBP transcriptional activity . However, recent studies also reveal that carbohydrate intake stimulates hepatic expression of activity , 41  , which hydrolyses circulating triglycerides into free fatty acids for replenishing triglyceride reserves . The ChRing site . Indeed,G2 cells . Addition levels . Angptl8α (Lxrα) , as wellSrebp1-c , 54.ANGPTL8 adenoviruses rapidly increased ANGPTL8 and triglyceride concentrations in plasma , which regulates many biological effects including embryonic development, cell migration, epithelial morphogenesis, inflammation, apoptosis and proliferation in different tissues ase, MET , 64. Whic islets , 66. Sarc islets Figure 1et genes . Since Het genes . In agren cells. . Altoget2.1.5via an interaction between liver and pancreas. PANDER is constitutively expressed and secreted by the liver in mice and humans, and increased under diabetic condition , was first described as a β cell cytokine facilitating insulin secretion under physiological conditions . Howeverondition . While pondition , Ratliffpromoter . Liver-se livers , suggestgulation Figure 1n vitro . Herein,nication .2.1.6Smoc1 was identified as a target gene of ChREBP. This was confirmed by ChREBP knockdown via small interfering RNA in vitro, which blunted Smoc1 expression in response to high glucose concentration , decreasing cAMP response element–binding protein (CREB) signaling in the liver  (via the interaction with the receptor for advanced glycation end products (RAGE). S100A6 caused alterations in the synthesis of intracellular cAMP and inhibition of mitochondrial ATP production in β cells. Knockdown of S100A6 was sufficient to improve insulin secretion in β cells from mice overexpressing hepatic ChREBP (via adenovirus infection)  , nuclearor (USF) , and NRFor (USF) , in respor (USF) Figure 1fection) . These r2.2via its direct interaction with ChREBP, FXR acting as a repressor on the ChoRE of glycolytic genes  also exhibit low circulating PAHSA levels, implying that systemic production of PAHSAs depends on ChREBP expression in adipocytes. Interestingly, a recent paper describes a novel secreted adipokine named WNT1-inducible signaling pathway protein 2 (WISP2/CCN5), which enhances adipose precursor cell proliferation  has been recently identified . Charactferation . Mice ovsitivity . In thisAdipoKO mice , an enzyme transcriptionally controlled by ChREBP (Excessive energy intake combined with a sedentary lifestyle imbalance the body homeostasis, causing an increase in the incidence of metabolic diseases, such as NAFLD, T2D and obesity . NAFLD iactivity . Indeed,teatosis  and in hteatosis , 101. Inteatosis . It has teatosis . In contteatosis . ChREBP crobiota . By the y ChREBP , reinfory ChREBP Figure 2de novo lipogenesis gene expression, as well as increased triglyceride content (Worth mentioning is that high fructose consumption strongly stimulates ChREBPβ, rather than ChREBPα, which is accompanied by increased expression of glycolytic and lipogenic genes in the liver of mice . ChREBPβ content . Finally content .Mttp or Tm6sf2, two genes regulated by ChREBP, impairs VLDL secretion, promoting hepatic steatosis and fibrosis (via a gut-liver axis.In addition to increased lipid synthesis, disturbances in lipid secretion also drive hepatic steatosis. The VLDL secretion rate appears to depend, not only on the availability of hepatic triglycerides, but also on the overall capacity for VLDL assembly. Indeed, liver specific deletion of fibrosis , 111. Infibrosis . Neverthfibrosis Figure 2fibrosis . In linefibrosis . These rdialogues, ChREBP appears as a key metabolic regulator of energy homeostasis.Most of the NAFLD patients are insulin resistant, rendering this disorder a strong risk factor for T2D development . It was 4Primarily known for its transcriptional role on glycolytic and lipogenic genes, ChREBP has emerged as a critical regulator of a broad range of processes across the body, with physiological and pathological implications. This review integrated the recent advances on ChREBP mediated inter-organ communication through the control of peptide and lipid production/secretion Figure 1All authors listed have made a substantial contribution to the work and approved it for publication. TC and CP wrote the manuscript. MR designed the figures. LP, MR, FB provided critical reading of the manuscript.


This study is aimed to understand the connections between didacticians’ meta-didactical praxeologies for the design and implementation of a teacher professional development program and their documentation work for the program itself. Didacticians are mathematics education researchers with the role of teacher educators. Data presents a case study, in which two didacticians (the authors) generate documents for the work with seventeen in-service mathematics teachers working at the lower secondary level, on inquiry mathematics tasks. The results reveal relationships, not yet fully addressed in research, between the intertwining evolution processes of the didacticians’ meta-didactical praxeologies and their documentation work, nurtured by the collaboration with the teachers. The whole process is led by the evolving goal of the professional development program, from the promotion of the classroom implementation of inquiry mathematics tasks to the broader goal of building an inquiry community with the teachers. This study could contribute to the introduction of an interpretative model for the didacticians’ work for the design and implementation of a teacher professional development program, based on the combination of Meta-Didactical Transposition and Documentational Approach to Didactics frameworks. During the last few years, a growing interest has been devoted to the study of teacher educators  can be situated within the broader frame of active learning and teaching  in the design process, which led to a continuous process of reconceptualization and redesign of teaching materials, as described in Jaworski .The transition between the two phases implied an evolution of the meta-didactical praxeologies of both the communities involved in the PD program, as we describe in the following sections, focusing on the didacticians’ side of this complex phenomenon. Pursuing critical alignment between didacticians and teachers was the leading goal of this evolutionary path.We use the MDT framework  mathematics teachers, started in 2017. It consists of two phases, attended by the same teachers: the first lasted for the first 3 years, while the second started with the fourth year, in 2020, and it will last 3 years too. In both phases, the synchronous activity for every year consists of 10 meetings of 2 h each, while the asynchronous phase consists of 10 h of individual work on the Moodle platform and 33 h of classroom experimentation. We will focus, in particular, on the first year of the second phase, a year of transition and evolution for the meta-didactical praxeologies of both the communities involved. The participants are 17 lower secondary school mathematics teachers, who have already attended the first phase of the PD program and have been informed that the second phase would have asked them for something more: a qualitative leap in their modalities of participation in the PD program entailing their direct involvement into the design of inquiry mathematics tasks. Eight participants have a master’s degree in mathematics or physics, while 9 have a master’s degree in biology, natural sciences, or agriculture. The majority of them are women, there are only 3 men. They are all experienced teachers, only 2 of them have less than 10 years of service, 9 of them have more than 15 years of service, and 5 of them have more than 20 years.The sheets provided to teachers by the didacticians during the meetingsThe slides projected by the didacticians during the meetings, to introduce the activitiesThe transcripts of the video recordings of the online meetings of the first year of the second phase. The video-recorded interactions occurred when teachers and didacticians were all together in the main session of the online meeting because in the separated sub-sessions, it was not possible to record, due to technical limitationsThe teachers’ protocols, provided in response to the didacticians’ requests. They include, for example, the design of inquiry mathematics tasks for students, teachers’ answers to reflection questions, or reports of classroom experimentationsThe data presented in this paper have been selected from a larger set of collected data. They derive from all the resources for teachers provided by the didacticians for the work during the first phase and the first year of the second phase of the PD program (the latter held online due to the COVID-19 restrictions). All these resources for teachers are retrievable on the web platform (Moodle) used for the asynchronous interactions between teachers and didacticians. In particular, we collected:Among the data enlisted in the previous section (points 1 and 2), we chose examples that seemed significant to us because they exemplify the didacticians’ meta-didactical praxeology in a specific phase of the PD program. They are both considered material parts of documents generated by the didacticians for the work during the PD program and as resources for the didacticians’ documentation work related to other, subsequent, documents. For all the analyzed documents, we traced the main resources on which the didacticians’ relied for their generation and the utilization scheme, composed of the class of situations, the rules of action, and the operational invariants.praxis (task and technique) and the logos components of the didacticians’ meta-didactical praxeology, referred to the phase in which that particular document was generated. In the “The DAD has developed a specific methodology, the reflective investigation of teachers’ documentation work, which gives a major role to teachers themselves. The active involvement of teachers, with a reflective stance, is needed because they have access to their documentation work and they can make visible some hidden resources :The didacticians, therefore, decided to integrate the “help cards” element in the requests contained in the activity-sheet of the 8th meeting, because they recognized it as an internalized praxeological element, both for the teachers and the didacticians themselves.We would list many other resources that contributed to the didacticians’ documentation work, as in the case of the first document described, but we chose to analyze in detail the most significant ones, in our eyes.The utilization scheme of the document for the 8th meeting of the second phase is summarized in Table logos of the same praxeology, but they are related to the specific context for which the document is thought.We can notice, as for the first phase, a strong relationship between the rules of action of the utilization scheme Table  of this This study aimed to address the issue of understanding how the evolution of the didacticians’ meta-didactical praxeologies for the design and implementation of a PD program is connected with their documentation work. In our case study, the results show that the evolution of the didacticians’ meta-didactical praxeologies from the first to the second phase of the PD program is reflected in the evolution of their documentation work. What emerged from the data analysis brought to the fore a complex intertwining of resources of diverse nature, both material and human, that contributed to the genesis of didacticians’ documents for the work with the teachers during the PD program.logos component), ready to be used by teachers with their students. In the second phase, the resources used and the utilization schemes of the didacticians’ documents testify to their choice of involving the teachers in the task design for their students. The analyzed documents, indeed, take into account teachers’ needs, their feedback, and the reasons for their design choices for students, coherently with the new task of the didacticians’ meta-didactical praxeology, which is fostering an inquiry community with the teachers. Critical alignment has been pursued by the didacticians basing the genesis of their documents on every possible source of information about teachers’ practices, motivations, and goals. In particular, the didacticians exploit as resources the teachers’ reports of classroom activities and the collective discussions during the meetings. We can call this kind of resources “human resources”  of two well-established theoretical frameworks, the MDT and the DAD, shows the internal connections between the two frameworks. It can, therefore, contribute to improving the understanding of teachers’ PD and also provide ideas for teacher educators’ PD.To sum up, the didacticians’ meta-didactical praxeologies referred to the design and implementation of the PD program evolved thanks to the joint work with the teachers, and this evolution is reflected in their generation of documents. Our study adds to the discussion on how didacticians’ design and implement a PD program, proposing a combined analysis with the MDT and the DAD frameworks, thus leading to a better understanding of their professional activity. In the resulting interpretative model Fig. , the didFurther research could be devoted to the application of the interpretative model presented in this paper to other types of praxeologies, for example, teachers’ meta-didactical praxeologies, in relation to their documentation work. Future studies could also apply it to the analysis of PD programs led by other didacticians, different from the authors, or in the case of teacher educators who are not researchers in mathematics education.


CYP2D6/tamoxifen, CYP3A5/tacrolimus, CYP2D6/opioids, DPYD/fluoropyrimidines, TMPT/thiopurines, CYP2D6/tricyclic antidepressants, CYP2C19/tricyclic antidepressants, NUDT15/thiopurines, CYP2B6/efavirenz, and CYP2C19/clopidogrel. In conclusion, although the global contribution of LAC countries remains low in the PGx field, a relevant improvement has been observed in the region. The perception of the usefulness of PGx tests in biomedical community has drastically changed, raising awareness among physicians, which suggests a promising future in the clinical applications of PGx in LAC.Pharmacogenomics (PGx) is considered an emergent field in developing countries. Research on PGx in the Latin American and the Caribbean (LAC) region remains scarce, with limited information in some populations. Thus, extrapolations are complicated, especially in mixed populations. In this paper, we reviewed and analyzed pharmacogenomic knowledge among the LAC scientific and clinical community and examined barriers to clinical application. We performed a search for publications and clinical trials in the field worldwide and evaluated the contribution of LAC. Next, we conducted a regional structured survey that evaluated a list of 14 potential barriers to the clinical implementation of biomarkers based on their importance. In addition, a paired list of 54 genes/drugs was analyzed to determine an association between biomarkers and response to genomic medicine. This survey was compared to a previous survey performed in 2014 to assess progress in the region. The search results indicated that Latin American and Caribbean countries have contributed 3.44% of the total publications and 2.45% of the PGx-related clinical trials worldwide thus far. A total of 106 professionals from 17 countries answered the survey. Six major groups of barriers were identified. Despite the region’s continuous efforts in the last decade, the primary barrier to PGx implementation in LAC remains the same, the “need for guidelines, processes, and protocols for the clinical application of pharmacogenetics/pharmacogenomics”. Cost-effectiveness issues are considered critical factors in the region. Items related to the reluctance of clinicians are currently less relevant. Based on the survey results, the highest ranked (96%–99%) gene/drug pairs perceived as important were  Several studies have estimated that more than 80% of the global population carries at least one actionable pharmacogenomic (PGx) variant, re known . This afre known . This fire known .A number of studies worldwide have demonstrated the cost-effectiveness of PGx testing, supporting its clinical application, particularly in public institutions . In thisThe application of PGx in the Latin American population presents several challenges, including the heterogeneity of the genetic admixtures, the socioeconomic context of each country and the differences in the health strategies implemented, among others. Arguably, the Latin American population is the most recently mixed and heterogeneous population worldwide, which reflects a history of massive settlement by immigrants (primarily Spaniards) and their variable admixture. In addition, the diversity of Amerindians since the Columbus voyages in 1492 and then the slave trade from Africa contribute to the heterogeneity. Today, Latin America has variable admixtures of African, European and Native American populations. The genetic variation of this region is underrepresented in genetic databases worldwide, and evidence related to PGx for these ethnicities is currently insufficient. This context is particularly relevant in PGx studies, where the phenotypic response to drugs is sometimes significantly different from that observed in homogeneous populations. Thus, Latin America can be seen as an ideal population for PGx studies in which polymorphic loci and linkage disequilibrium can be used to infer the genetic basis of drug response .Despite several individual and collective efforts in LAC to implement PGx in clinical practice, the transition represents a relevant challenge due to genetic/pharmacogenetic diversity, political variables and idiosyncratic perspectives. The perceived importance of barriers to implement PGx testing in clinical practice, examined in 2014 for LAC, as all over the world, seems to be still very relevant, i.e., a) the need of clear guidelines for the use of PGx in clinical practice, b) the insufficient awareness of PGx among clinicians, and c) the absence of regulatory institutions that enables the implementation of pharmacogenetic tests .www.solfagem.org). This society aims to strengthen the development of PGx scientific research, both theoretical and experimental, to increase the awareness and promote the dissemination of the discipline through knowledge creation, clinical tool searches, and the identification of products or biomarkers that can improve current disease treatments, as well as any other initiative conducive to a broad utilization of this scientific discipline for the benefit of Latin America and the Caribbean (LAC) and global public health. In 2015, the first president of SOLFAGEM became a member of CPIC to represent LAC countries and learn about the CPIC guidelines and procedures. However, no review of CPIC guidelines or contributions has been requested from any of SOLFAGEM’s presidents up to this point. Afterwards, in March 2019, the Ibero-American Program on Science and Technology for Development (CYTED) network called RELIVAF , with the participation of 12 countries, was born. The primary goal of this organization is to create a network of Latin American basic-clinical research centers in PGx to establish an exchange and to generate discussion groups about the configuration of specific (customized) clinical pharmacogenomics implementation Guidelines for LAC, according to ethnic variability, to be used in regional health systems.In 2014, a group of Latin American researchers was established to address the problem , and in In this respect, given that the majority of the authors first examining PGx testing and the barriers to clinical application in 2014 are now members of RELIVAF, we present here an updated assessment of LAC after 8 years of joint efforts to improve the PGx knowledge and its clinical applications. In addition, a renewed exploration of the contribution of Latin America to global PGx knowledge is included.To understand the evolution of the PGx field in LAC and the contribution of LAC countries to global PGx advances, a search was conducted for publications from 1 January 1959, to 1 November 2022, using the Scopus database and other complementary sources  with the following PGx-related keywords: polymorphisms, pharmacogenetics, pharmacogenomics, SNPs, biomarkers, adverse drug reactions, precision medicine, personalized medicine, and clinical pharmacology. The keywords were used in a logical combinatorial manner to ensure they adjusted to the field. First, we  analyzed abstracts and then, when necessary, the full text of manuscripts, to ensure their cover topics within the field. This approach yielded 38,786 publications worldwide and 1,337 from 20 LAC countries (starting in 1984). Publications from Antigua and Barbuda, Dominican Republic, Guyana, Paraguay, Suriname, and Trinidad and Tobago were not found . CompletUsing the results of the publication’s search and complementing this with members of LAC scientific associations and societies  and the authors of this manuscript, we identified 188 potential respondents, who were contacted to answer a structured survey  in ordert-test with unequal variance (STATA 17.0) using number of responses, average and standard deviation of each survey and p < 0.05 as statistically significant  and the average ± SD of all respondents for each barrier is reported . Second,nificant .Our search for PGx publications revealed that the field is a hotspot research area, giving rise to almost 39,000 publications since Dr. Friedrich Vogel’s first publication, where the term pharmacogenetics was coined and defined . Twenty p < 0.001); and 6) “need for demonstration of clinical validity and utility of pharmacogenetic/pharmacogenomic tests” (8.55 points), which is significantly higher than the 2014 survey (6.92 points) (p < 0.001). Additionally, the “Reluctance of clinicians to use pharmacogenetic markers” and the “insufficient concern about pharmacogenomics among clinicians” were significantly lower in the present survey compared with 2014 . Conversely, the “Ethical, legal and social implications of pharmacogenomics” and the “Concerns about the test costs” were significantly higher than in 2014 .The results observed in CYP2D6/tamoxifen (99), CYP3A5/tacrolimus (99), CYP2D6/opioids (98), DPYD/fluoropyrimidines (98) and TMPT/thiopurines (97), CYP2D6/tricyclic antidepressants (97), CYP2C19/tricyclic antidepressants (97), NUDT15/thiopurines (97), CYP2B6/efavirenz (96), and CYP2C19/clopidogrel (96) . These pairs include the same 51 pairs evaluated in 2014. However, some are now grouped by CPIC . Thus, the number of CPIC pairs does not perfectly match between the surveys , the higrel (96) . With rerel (96) .The PGx field has been growing since the first contributions of Drs. Friedrich Vogel and Arno Motulski . Howeverhttps://relivaf.mailchimpsites.com). Besides, the setting-up of two specific laboratories in Guatemala and Colombia by RELIVAF members , a Guideline for PGx studies in Cuba . One international RELIVAF-Latin American postgraduate program recently finished  in the present survey. This indicator reflects significant concern about the feasibility of implementing PGx in LAC, which agrees with the results of the item “concern about test costs,” which increased by 1.00 point from the 2014 to the 2022 survey (7.48–8.48). In developing countries, the cost factor is a persistent obstacle in the implementation of new technologies. It is generally believed that if it is possible to demonstrate the cost effectiveness of PGx implementation in daily clinical practice, the effect of other obstacles derived from cost effectiveness would be mitigated, such as the lack of its promotion in health systems.However, significant advances should be noted; for example, the “reluctance of clinicians to use pharmacogenetic markers” is now less relevant, decreasing significantly from 6.7 points in 2014 to 5.6 in 2022. This change may be related to the knowledge and awareness of the field, which is supported by both the significant decrease in the “insufficient concern about PGx among clinicians” from 8.5 points in 2014 to 7.3 points in 2022 and the significant increase in the concern about the “ethical, legal and social implications for the implementation of PGx” from 6.0 points in 2014 to 7.6 points in 2022. In addition, the “insufficient use of electronic records information on patients” decreased from 7.8 points in 2014 to 7.3 points in 2022, although it is not significant it reveals a trend toward an improvement during these years. The increased awareness of clinicians may in turn encourage regulatory institutions to set up guidelines and regulatory documents in the PGx field.All of the above identified more relevant barriers in the LAC region overlap with barriers identified in other regions of the world, with some highlighted differences, especially in relation to developed countries . Cost isAccording to the survey results concerning barriers, the clinical demonstration and validation of PGx tests are key points for implementing PGx in LAC. Therefore, in the present survey, the inclusion of 54 gene/drug pairs was performed to evaluate concerns about the clinical utility of these PGx biomarkers . Forty-tCYP2D6/tamoxifen, CYP3A5/tacrolimus, CYP2D6/opioids, DPYD/fluoropyrimidines, TMPT/thiopurines, CYP2D6/tricyclic antidepressants, CYP2C19/tricyclic antidepressants, NUDT15/thiopurines, CYP2B6/efavirenz, and CYP2C19/clopidogrel; only in the TMPT/thiopurines for CPIC pairs was also found on the 2014 survey. Notably, all CPIC pairs were highly ranked by 77–99 respondents, with the lowest ranked pair being COMT/opioids  on the 2014 survey. This finding is likely a reflection of the improvement in the PGx scientific information available in our countries, giving rise to a better estimation of the importance of gene/drug pairs. The highest ranked gene/drug pairs in psychiatry and oncology suggest that LAC countries deal with similar health problems to developed countries.According to the survey results, the highest ranked (96%–99%) gene/drug pairs perceived as important were /opioids . With reperidone , and allAlthough the pairing of CYP2D6/SSRIs (selective serotonin reuptake inhibitors) was not one of the best-evaluated pairs, its importance has increased significantly from 15 in the 2014 survey to 85 in the 2022 survey . This maSome limitations of this survey must be noted. First, even though the sample is considerably higher than in the 2014 survey and is probably more representative, it is nonetheless still small. Therefore, a future larger survey may be required for better results. However, the study participants from each country are people with at least basic knowledge in PGx. Second, the perceived importance of the gene/drug pairings could be different in each country of the region due to different access based on political, socioeconomic or administrative differences. Finally, the lack of sufficient cost–efficacy evidence of PGx, technical limitations in some countries and ethnic profiles of specific countries may significantly impact the implementation of PGx in clinical practice.The implementation of technologies that involve target sequencing or in-house kits may represent a more affordable option for the region. In this respect, it is necessary to guarantee the identification of the most frequent genetic characteristics for these populations. However, once this main challenge is successfully overcome, the region will significantly benefit from precision medicine and public health. Collaborative work can provide meaningful information related to native and admixture PGx characteristics for each region, given the rich and variable response of LAC countries’ mixed populations.Nowadays, even though the PGx capacity in LAC is increasing, this increase is at a similar rate to the global community, thus the contribution of Latin American and Caribbean countries to the PGx field worldwide remains limited. Our work shows improvement in both the number of publications as well as in the spread of knowledge and institutional efforts to address PGx as a relevant tool of personalized medicine in the region. Interestingly, the perception of the usefulness of PGx has drastically changed, raising awareness among physicians, which reflects a promising future in the clinical applications of PGx. The development of public policies and the implementation of intersectoral action plans that involve the different stakeholders of the health system continue to be a challenge in the implementation and growth of precision medicine in LAC.


The majority of childhood, Adolescent and Young Adult (AYA) cancer survivors suffers from long-lasting health issues following cancer treatment. It is therefore critical to explore effective health promotion strategies to address their needs. Exposure to nature is a promising approach to support the needs of young cancer survivors. This study investigated whether it is feasible to conduct a randomized controlled trial (RCT) of a wilderness program for childhood and AYA cancer survivors.Eligible participants were aged 16–39 years, had a cancer diagnosis, and met minimal criteria. Seventy-one individuals expressed interest and 59 were randomized to either a wilderness or a holiday program. The wilderness program involved an 8-day expedition including backpacking, sea kayaking, gorge climbing, camping, bush-craft skills, and mindfulness-practices. It was followed by a 4-day basecamp after 3 months. The comparison was an 8-day holiday program at a Spa-hotel followed by a 4-day holiday program at the same hotel after 3 months. Primary outcome was study feasibility and safety.p = 0.024). Program adherence and data completeness was high (> 90%) in both arms. Adverse Effects (AEs) in the wilderness vs. the holiday program were similar . The most frequent AE was tiredness, all were mild to moderate in severity, and serious AEs were not reported. Nature connectedness significantly increased over time in the wilderness program participants, but not in the holiday program (p < 0.001). No differences were found between the two study arms regarding quality of life, self-esteem, or self-efficacy.Ultimately, 19 AYAs participated in the wilderness and 23 in the holiday program. All completed the study at one-year follow-up. Participants were mostly female (70%) and represented diverse cancers. Clinical characteristics were similar between study arms excepting greater age at cancer diagnosis in the wilderness program .The online version contains supplementary material available at 10.1186/s12889-023-16408-x. In the past decades, five-year overall survival rates for childhood and Adolescent and Young Adults (AYA) cancer survivors have increased to 86% –3. This It is of utmost importance to provide childhood and AYA cancers survivors with effective interventions and rehabilitation- or health promotion programs during their survivorship to support them in meeting physical, psychological, and social challenges. A systematic review and meta-analysis of psychosocial interventions for AYA cancer survivors demonstrated that technology-based as well as psycho-educational, multi-component and physical exercise interventions have very small and non-significant effects in this target group . Most ofGiven the multitude of chronic health issues among childhood and AYA cancer survivors and given that there are few interventions that have the potential to impact on multiple factors, we developed and piloted a multi-component health promotion intervention: The Wilderness program for childhood, Adolescent and Young Adults (WAYA) program . AdventuExposure to nature has shown to positively affect our health and well-being , 33, invThis study was a pilot RCT with a two-armed parallel-design using individual randomization. The protocol for this study was registered in a public trial registry  and previously published . The stun = 1683) or the Swedish Childhood Cancer Fund (n = 300). A short announcement of the planned study was posted in e-newsletters and social media (Facebook) of these cancer organizations. The announcement was an open invitation for childhood, AYA cancer survivors aged 16–39 years to participate in a pilot study that aimed to compare participation in an outdoor program with participation in a holiday program regarding possible benefits for their health and well-being. Those interested were referred to the WAYA study website of Mid Sweden University [Individuals eligible to participate were AYA’s aged 16–39 years who had received a cancer diagnosis during childhood, adolescence, or young adulthood. No eligibility restrictions were placed on the type of the individuals’ cancer diagnosis or time after cancer treatment. Participants had to be capable of walking at least 2 km without a pause, for which walking aids were allowed. Exclusion criteria was an active cancer treatment or medical condition that prevented safe travel and participation in the study, as evaluated by their treating physician/oncologist. Participants were recruited among a total of about 1983 childhood, AYA cancer survivors who were members of the Swedish cancer organization Ung Cancer  Theoretical foundation in the ecosophy of the Norwegian philosopher, mountaineer and environmental activist Arne Næss ; 2) ConcNature in the WAYA program had both a contextual and active role in support of the health and well-being of childhood and AYA cancer survivors, wherein their health was regarded to be intrinsically interwoven in a bi-directional fashion with the health and well-being of nature . ParticiThe structure of the wilderness program is graphically depicted in Fig. Extensive discussions with researchers, patients and public representatives led to the design of a holiday program as an appropriate comparison arm. A holiday program was aimed to control for attention, group support and separation from home and job life, all factors that may positively influence the health and well-being of childhood and AYA cancer survivors . ActivitActivities in the holiday program included, amongst others: Spa, fine dining, watching movies, reading, games, and visiting museums. Guided group gatherings took place twice a day  involving talking with and listening to how everyone was doing. The holiday program did not include guided nature, outdoor or structured physical activities.Just like the WAYA program, the holiday program consisted of three parts: 1) An eight-day holiday; 2) A three-month period in between where participants were contacted twice to discuss how they were doing; and 3) A four-day holiday  needed to recruit at least 40 eligible participants that were willing to be randomized. To obtain further insight regarding recruitment and inclusion of participants in the study, the following parameters were calculated: Recruitment rate: Calculated as the number of individuals who contacted Mid Sweden University expressing interest in the program compared with the estimated maximum number of individuals who could have read information on the study via Facebook and newsletters from the national cancer organizations; Eligibility rate: The number of individuals meeting the eligibility criteria divided by the total number of individuals who contacted Mid Sweden University; Enrollment rate: The number of individuals randomized into the study divided by the number of eligible individuals; 2) Program preference and expectations. Program preference  and expectations (open question) were directly asked to eligible participants during the first contact; 3) Willingness to be randomized. This was asked to eligible participants at the first contact (yes/no) and calculated based on the number of participants that dropped out after randomization because they had hoped to be allocated to the other program or those that admitted after randomization that they would have dropped out if they had been allocated to the other program; 4) Attrition rate before, and after start of the intervention: Calculated as the number of individuals who dropped out of the study before and after study start, divided by the total number of enrolled participants; 5) Adherence to the study program, calculated as the number of intervention days on which participants attended the program-offered activities, divided by the total number of intervention days when program activities were offered; 6) Data collection completeness, calculated as the % of completed questionnaires divided by the total number (no.) of questionnaires to be completed as part of the study .Primary study outcomes were study feasibility and safety. The following feasibility parameters were selected for this RCT: 1) Adverse Effect (AE) [Regarding safety, the occurrence of adverse events was closely monitored during the study interventions. An adverse event was defined as any incident, accident or another unfavorable or harmful outcome that occurred during or after the intervention but was not necessarily caused by the intervention . In caseect (AE) . In bothect (AE) . Verbatiect (AE) . Adverseect (AE) . ExposurSecondary study outcomes in this RCT were the following self-reported questionnaires:Nature Relatedness Scale (NRS): A 21-item scale measuring ones ‘affective, cognitive, and physical relationship’ with nature [h nature . Since tMinneapolis Manchester Quality of Life instrument (MMQL), a seven quality-of-life domain questionnaire for AYA cancer survivors [urvivors , validaturvivors .Rosenberg Self-Esteem Scale (RSES), a 10-item scale measuring global self-worth [lf-worth , and vallf-worth .Generalized Self-Efficacy scale (GSE): A 10-item scale measuring general belief in oneself to solve problems and reach goals [ch goals , also trch goals .Changes in secondary outcome measures were analyzed according to a pre-defined data analysis plan. Across all scales used to assess health-related outcomes, higher scores reflect more positive outcomes.Participants were individually randomized to either the wilderness or holiday program by means of a Random Allocation Software Program using a random block size of two. Three age groups  and gender  were used as stratification factors to achieve an equal distribution between the two arms. Separate randomization lists were generated per strata. Allocation concealment was guaranteed since the randomization and assignment of participants to the two arms were performed independently by the first author (MCJ), who was not involved in participant recruitment. Although it was not possible to blind for the intervention, participants were informed that the study aimed to investigate two possible effective interventions, not revealing which intervention was hypothesized to be the most effective one. Furthermore, the statistical outcome analyses were performed by the second author (HD) who was blind to program allocation.Figure n = 20 in the wilderness program and n = 20 in the holiday program. This sample size was chosen following the recommendations of Whitehead et al. (37), to detect a small to medium effect size for changes in quality of life.We aimed to include a total of 40 participants, Safety analyses were performed on the safety analysis set and included all individuals that had participated in one of the program interventions for at least one day. A safety code book and a safety analysis plan assisted in the analysis. Crude (unadjusted) Relative Risk (RR) with 95% Confidence Intervals (CI) were calculated to indicate the risk of having an AE adjusted to days of exposure in program interventions. Pearson's chi-squared test was used to calculate differences in AE severity between the two study arms. Chi-squared statistics and independent samples t-tests were used to calculate group differences in socio-demographic variables and cancer characteristics. The Alpha level was set to 0.05.p-values < 0.05 to indicate significant differences. Statistical analyses were performed using SPSS Statistics version 28. Cohen’s d effect sizes were manually calculated using the mean difference (the beta) divided by the standard deviation, obtained from the standard deviation (SD) of the residuals.To explore possible change over time in nature-relatedness, quality of life, self-esteem, and self-efficacy scores within study arms, linear mixed regression models with fixed effects were used. The covariance matrix was set to A1 Heterogenous which assumes that the covariance between time points gets weaker as measurement times grow further apart. To explore possible changes between study arms, a mixed-effects regression models with random intercepts were conducted. Missing data were imputed by means of multiple imputation . Scores n = 3), or because of health problems (n = 1). Therefore, a total of 62 individuals were screened at the first contact interview and met the eligibility criteria.The CONSORT flow diagram is depicted in Fig. n = 2) or not wanting to be the youngest participant (16 years) in the study (n = 1). Ultimately, a total of 59 eligible individuals were randomized to either the wilderness program (n = 29) or holiday program (n = 30) . This was reflected by a higher percentage of AYA cancer survivors in the wilderness compared to the holiday program   Table . At basey) Table . Most fry) Table . Hormoney) Table .n = 7 in the wilderness arm and n = 5 in the holiday arm), did not significantly differ from those that received the allocated interventions .The demographic characteristics of the 12 randomized participants that were met at contact moment two but did not receive the intervention  Table . The numn = 15) of individuals had no preference for either one. Preference of participants for either the wilderness or holiday program did not significantly differ between the two arms in the study (p = 0.426). Before study start, individuals expected the wilderness program to be challenging (n = 24), a place to meet and talk with others that have gone through the same (n = 19), to experience something new (n = 17), and to get energy from being  out in nature (n = 17). Expectations about the holiday program were mostly time to relax (n = 47), to meet and talk with others (n = 26), nice to be away (n = 10), risk to get bored (n = 9), and time for reflection (n = 5).Program preference and expectations: Out of 62 individuals that were screened for eligibility during the first contact, most (58.1%) preferred to participate in the wilderness program Table . In 17.7Willingness to be randomized: Although all 62 individuals agreed to be randomized at the first contact, it turned out that one individual who had a strong preference for the holiday program dropped out of the study because she had been allocated to the wilderness program. Another participant who ended up in the holiday program confessed that if she had been randomized to the wilderness program, she would have dropped out. Willingness to be randomized was thus 60 out of 62 individuals (96.8%) Table .Attrition rate: Out of the 59 individuals initially enrolled, a total of 17 dropped out after randomization but before start of the intervention. The attrition rate before the start of the intervention was thus 28.8% in total, 34.5% (10/29) in the wilderness and 23.3% (7/30) in the holiday program. Since no participant dropped out of the study after entering the intervention, neither in the wilderness program nor in the holiday program, the attrition rate after start of the intervention was 0% overall and, in both programs (Table n = 1) and an unexpected illness in the family (n = 1). In the holiday program, one participant in the first group discontinued intervention after four days because she suddenly had to start a new summer job  were labeled as AEs since they were certain, probable/likely, or possibly related to one of the program interventions. In the wilderness program, 17 out of 19 participants reported a total of 113 AEs. In the holiday program all participants (n = 23) reported at least one AE, with a total of 141 AEs during the whole study period. The RR for the occurrence of AEs in the wilderness program was not significantly higher than in the holiday program (RR: 1.0 (CI 0.8–1.3))  in the younger age group . Slightly higher RR with significant differences were observed between the two study arms for the variables program part, gender, age, and survivorship (Table AEs were of mild or moderate severity (grade 1–2), all AEs were resolved during the study period Table  and no sip Table . Howevern = 1). Frequently reported AEs by participants in the holiday program were problems to fall asleep (sleeplessness), to stay asleep (insomnia) and specific brain fatigue (mild neurocognitive disorder). The most frequent MedDRA SOC groups of AEs in both the wilderness and holiday program were general disorders and administration site conditions and psychiatric disorders (Table As shown in Table rs Table . The lisp < 0.001). No significant changes over time were observed among participants in the holiday program . P-values of within study arm differences are not shown in the Table p < 0.001) with a medium Cohen's d effect size of 0.63 . No differences between study arms were observed for the other MMQL domains (results not shown). No significant differences in MMQL, RSES or GSE scores over time were found at any time-point within the wilderness program. There were significantly lower RSES scores at T2 compared to T3 within the holiday program , and significantly lower GSE scores at T1  and T2  compared to T3 within the holiday program. MMQL scores over time within the holiday program were not significantly different , self-esteem (RSES) and self-efficacy (GSE) Table . Furthernt Table .The present study shows that it is feasible to perform a randomized-controlled study design on a complex intervention such as a wilderness program and with a medically vulnerable study group such as childhood and AYA cancer survivors. Within half a year, the number of participants needed for this pilot study was recruited and almost all childhood and AYA cancer survivors were willing to be randomized. These results are thus in sharp contrast to the previous study by Gabrielsen et al. , where iAnother important finding of this study is that a supervised wilderness adventure is just as safe for childhood and AYA cancer survivors as enjoying a holiday program at a Spa hotel. The relative frequency of reported AEs was similar in both study arms, and all AEs were non-serious, mild, or moderate of severity, and resolved during the study period. Reports of tiredness, mosquito bites, situational anxiety, and constipation by participants in the wilderness program is to be expected when exposed to a wilderness expedition, challenging physical activities such as sea-kayaking and rock climbing, and outdoor food and no access to a bathroom. Despite the taxing wilderness conditions, it was interesting that tiredness was also frequently reported by participants in the holiday program. Whereas participants in the wilderness program had full days of physical activity in nature, participants in the holiday program were mostly relaxing, sleeping and enjoying good food. It may be that more vigorous outdoor physical activity resulted in higher quality sleep, increased group connectedness, and greater well-being, albeit, not statistically different between study programs. It is noted that sleeplessness and insomnia were more often reported by participants in the holiday program.Although AEs were commonly reported by participants during both programs, it should be noted that several of these were to a greater or lesser extent related to their existing long-term health issues prior to study start , it appeared that the overall recruitment rate was very low (3.6%). It is likely that the number of cancer survivors exposed to the study outreach was much lower than the total number of people who were signed up with the two cancer organizations. Furthermore, response may have been lower due to COVID-19 concerns. Other recruitment strategies to supplement members of national cancer organizations may be needed in order to reach out to this population. Health care, educational and cultural systems are quite similar in the Nordic countries. It is therefore recommended to perform a future larger multi-center RCT in close collaboration with centers in Denmark, Finland, and Norway. Another recommended strategy for recruitment of participants is to collaborate with hospitals in future trials.Another important observation was that a substantial percentage of eligible individuals dropped out before study start (attrition rate 28.8%) because of lack of time or not being able to get free from work. Previous studies have published difficulties in recruiting AYA cancer survivors , 26. ThiIt was intended to recruit both adolescent and young adults for participation in the present pilot study. To our surprise, it was only young adults from 20 years and older that participated. Only one eligible adolescent showed interest to participate but declined participation when she found out that she would be the youngest in the group. Since both national cancer organizations through which participants were recruited have adolescents among their members, it appears that such programs mainly appeal to young adults. The WAYA program was specifically developed for AYAs aged 16 to 39 years. This allowed for participation of individuals that were diagnosed with cancer throughout childhood, adolescents, and young adulthood. We previously reported that inclusion of a diverse group of young cancer survivors in a wilderness program seems to be of added value for group bonding and learning purposes . The conThe choice for the secondary health-related outcome variables quality of life, self-esteem and self-efficacy in the present study was based on the results of a scoping review that mapped the concept, content and outcome of wilderness programs for childhood cancer survivors . This scIn conclusion, the findings of this present pilot study demonstrate that it is feasible to conduct a RCT on a multi-component health promotion intervention such as the WAYA wilderness program. Both the wilderness and comparison holiday programs were acceptable for childhood and AYA cancer survivors. Serious adverse effects related to program participation were not reported and remaining effects were mild to moderate in severity and resolved within the study period. It was demonstrated that participation in the wilderness program was just as safe as spending vacation time at a Spa hotel. The findings of this study underscore the necessity to develop and apply other recruitment strategies in order to recruit sufficient participants for a future larger RCT. In the planning of such a large RCT it is recommended that the program content be adapted to stratify for the inclusion of cancer survivorship (childhood cancer versus AYA cancer), and to select health-related outcome variables based on the qualitative analysis of the lived experiences of participants in the program. This study provides critical and useful information for public health actors and outdoor/wilderness facilitators that plan to undertake wilderness expeditions with childhood and/or AYA cancer survivors. In addition, it informs researchers how to design a future larger RCT with the ultimate goal of investigating the effectiveness of wilderness programs on the health and well-being of childhood and AYA cancer survivors.Additional file 1.