This study sought to evaluate the cultural adequacy of materials and procedures of the Strengthening Families Program (SFP 10-14-UK) and to identify requirements for its cultural adaptation to Brazilian families. The descriptive study had 33 informants, including external observers, managers, multipliers, facilitators, adolescents, and parents. The data were collected at a pilot application in the Federal District. Direct observation was applied to four intervention groups, with seven meetings of 150 min for families, parents/guardians and adolescents, and mixed nominal groups at the end of the interventions. The results, analyzed through content analysis and descriptive statistics, provided evidence that SFP was perceived as sufficiently appealing, culturally relevant, and partially clear. Recommendations for cultural adaptation of linguistic aspects of the materials and procedures were made, considering the cultural and educational differences of the participant families. Focus on implementation quality, including infrastructure, families’ mobilization and continuous planning, was recommended. Replication studies in other Brazilian regions and analyses of contextual and political dimensions are suggested. Some of the main risk factors to Brazilian adolescents’ health are alcohol abuse, use of illicit drugs, risky sexual behavior, violence, and parental negligence  . The starting point for the implementation of SFP was in the context of the Comprehensive Family Attention Program. This service integrates The National Policy of Social Assistance , which joined the program in the beginning of 2014, and the Ministry of Justice , which joined the program by the end of 2014, effectively participating in its implementation in the beginning of 2015 , who received SFP training from the researchers responsible for the English version of the Program adopted in Brazil; mothers (n = 4), fathers (n = 2), and adolescents (n = 7) who were participants in Strong Families Program; facilitators (n = 6) who implemented the Program’s sessions; federal multipliers (n = 6) who were trained to act as creators of new implementation teams; and public managers (n = 4) with ties to the Ministry of Health. The external observers were recruited from researchers at the University of Brasilia; the facilitators, mother, fathers, and adolescents were recruited from the Coexistence and Relationship Strengthening Services, the Social Assistance Reference Centers , and the Legal Guardianship Council (a service focused on the protection of the rights of children and adolescents); and the multipliers and public managers were recruited from the Ministry of Health team. All the participants of this study were properly informed about their rights and expressed their agreement to participate in this study via an informed consent form. They were told about the voluntary character of the participation, the objectives of the research, the possibility of desisting at any moment, the protection of their identities in the data analysis and publications, access to the study results, anticipated uses of the data, the minimal risk, and the potential benefits of the study.The study design was descriptive, with qualitative and quantitative measures and multiple informants. There were 33 informants, comprising external observers  and the other for adolescents, both lasting approximately 60 min. The second was held jointly, i.e., with both the adolescents and their parents/guardians. The participants were offered refreshments at the beginning, end, or between the first and second parts of the meetings according to their wishes at the time. Six federal multipliers and 15 social educators from the Coexistence and Relationship Strengthening Services, the Social Assistance Reference Centers, and other professional entities associated with the Federal District acted as facilitators. These professionals held degrees in psychology, social work, and history. The federal multipliers were professionals selected by the Ministry of Health to facilitate the program and support the process of applying the intervention in the pilot phase as well as to form new implementation teams in the future phase of spreading the program through Brazil.The content addressed in the meetings, provided in a manual translated beforehand by technicians of the Ministry of Health , relevance , and clarity (to what extent were the procedures carried out in a way that was understandable to the participants?). For low scores , it was requested that the observer indicate what could be changed to improve the cultural appropriateness of the SFP for Brazil with open questions such as: “What must change to make the session more coherent for the local culture?”. At the end, the observer was requested to openly and honestly record impressions of the weak and strong points of the sessions.Direct observation of the seven meetings carried out in the four groups of the SFP was done. During the first hour of the intervention, an observer dedicated himself to the observation of the group of adolescents, while a second observer observed the group of parents/guardians. In the second hour, both observers followed the joint meeting. The observers were instructed to adopt a non-invasive behavior during the sessions and to make notes in the field diary immediately after the joint meetings.Based in a procedure developed by Allen et al., , the ScrThe Script for Nominal Groups with Multiple Actors was used 1 month after the end of SFP, when three mixed nominal group meetings were held, with the participation of parents/guardians, adolescents, facilitators, multipliers, and public managers to assess elements to be adapted in the program targeting its appeal and cultural relevance to Brazil. The groups represent four low-income regions of the Federal District where the SFP was carried out. Mixed nominal group 1 comprised 12 participants: two female parents/guardians, three adolescents (two boys and one girl), three facilitators, two multipliers, and two public managers connected the Ministry of Health. Mixed nominal group 2 comprised nine participants: two female parents/guardians, two male adolescents, two facilitators, two multipliers, and one public manager. Mixed nominal group 3 comprised eight participants: two parents/guardians, one adolescent, one facilitator, two multipliers, and two public managers.The procedure described by Allen et al.,  was emplThe quantitative data derived from the field diaries of the external observers’ judgments regarding clarity, relevance, and appeal of the meetings were analyzed with descriptive statistics (frequency) using the software SPSS v.20.0, and graphics were generated for visual inspection Fig. . The quaThe cultural appropriateness of the procedures performed was evaluated by external observers according to three criteria: appeal, cultural relevance, and clarity. According to the external observers, the first sessions were quite appealing and reasonably appealing, the middle sessions were quite appealing, and the final sessions varied between somewhat appealing and highly appealing. No session was classified by the external observers as unappealing. According to Fig. As for the relevance of the procedures to the local culture, those adopted in the initial and middle sessions were considered to have “much to do with the local culture” by the observers. In the end sessions, cultural relevance varied between “reasonably related to local culture” and “everything to do with the local culture”. No session was evaluated as having “nothing to do with the local culture” or “little to do with the local culture” see Fig. .While the appeal and cultural relevance of the procedures were considered appropriate by the observers, with relative homogeneity between sessions, the clarity of the procedures varied greatly over the program’s implementation, ranging from insufficient clarity (unclear) to sufficient (completely clear). The unclear sessions, which are at greater risk of being misunderstood by the participants, occurred at the beginning, middle, and end of the program. Nonetheless, sessions were most commonly reported as having a proper degree of clarity, “quite clear,” or “completely clear,” as seen in Fig. Although the external observers offered specific suggestions for improving the appeal, cultural relevance, and clarity, the data will be presented as a set. These suggestions, presented with the intention of making SFP more motivating, culturally relevant, and understandable to the participants, fall along two lines: cultural appropriateness and implementation quality.In the first category, adaptations to the language of the activities and materials of the SFP were suggested. Suggested language adaptations included changing the names of cars, games, objects, and places as well as typically English day-to-day activities for others common in Brazil: for example, replacing “family crest” for “family flag.” Other suggestions for activities included the use of strategies that do not require reading or writing, as a significant number of the adults were illiterate and younger children presented difficulty reading, and customizing the playful activities to the developmental necessities of the younger children (10) and older adolescents (14). Finally, examples of suggested adaptations for materials include improving the quality of dubbing and sound of the DVDs, including extended family—beyond grandparents—who might offer support in the branches of the “family tree” activity (session 2), thus taking into account multiple family configurations and adapting the activities that require reading, as the illiteracy of the parents/guardians affects the engagement and length of the activity, compromising fidelity.In the second category, implementation quality, greater attention to planning, introduction, and conclusion of the activities was recommended, as well as to the facilitator’s ability, time management, and group management. Even though such recommendations are not directly related to cultural adaptation, the external observers considered them capable of impacting the understanding and appeal of the procedures for the participants. It was suggested that the sessions be planned together by the team of facilitators to advance everyone’s performance quality, improve the distribution of functions, and facilitate uniform understanding among facilitators, which should impact the clarity of the instructions given to the group. It was also suggested that the session’s material be prepared in advance, as too much time was spent writing information on the board, which allowed the participants to get distracted. To boost the understanding and meaning of the procedures, it was suggested that the objectives and relevance of the activities be explained and concluded, summarizing its main aspects and improving the transference of the reflections and lessons to day-to-day interactions, before moving on to the next planned activity. The external observers suggested the facilitators improve their communication abilities, such as using understandable language in place of technical jargon (such as “empowerment”), using examples, and giving the group feedback through empathetic verbalization and paraphrasing. Regarding time management, synchronizing the parent/guardian sessions and adolescents to end them at the same time and avoid delays in the family sessions, which might reduce the participants’ interest, was made. Finally, as for group management, boosting interaction between participants and the group’s affective closeness was suggested.The necessities for adaptation ranked by the three mixed nominal groups, comprised of adolescents, parents/legal guardians, facilitators, multipliers, and public managers, are described in Table As the external observers proposed, the participants of the mixed nominal groups suggested reducing the written activities, given that many participants of SFP cannot read or do not read fluently, which can cause feelings of discomfort and alienation from the group, beyond prolonging the session, demotivating the participants, and damaging fidelity. Regarding the DVD quality, improvements were suggested regarding the actors , scenery , language , content , and audio .The necessity of making the language more relevant to the local Brazilian culture  and improving non-verbal aspects (such as the emotion expressed by the intonation) was indicated. Among the linguistic adjustments, the replacement of places, musical instruments, cars, and names with Brazilian names or nouns familiar to the local reality (for instance change “clarinet” for “acoustic guitar”) was suggested.Finally, it was the groups’ consensus that the process of the announcement, recruitment, and mobilization of families must be improved. It was recommended that the strategies of the announcement and presentation of SFP to the families before the first session be broadened. Such measures were considered to have the potential for boosting the attraction and adherence of the families. Even though not directly focused on cultural adaptation, these were considered relevant for advancing the perception and appeal of SFP and motivating participation.Two mixed nominal groups pointed to the need to offer distinct activities for children and older adolescents and to extend the duration of SFP to make time management more flexible and to allow more families the opportunity to participate. Other suggestions made by only one of the mixed nominal groups can be seen in Table Among the ideas proposed in Table In the same vein, the facilitators reported that the execution of the SFP in the Coexistence and Relationship Strengthening Services improved the functioning of the service by offering a method of intervention coherent with the general objectives of the service , that is, the strengthening of family ties. According to their perception, it acts as a guideline for their efforts: “The program worked super well! It explored the work with families. I was able to carry out the work. Fantastic!” . In summary, participants and facilitators reported a positive perception of SFP’s social validity, even though it needed further cultural adaptation and improvements in implementation quality.This study’s main objective was to evaluate the cultural adequacy of the materials and activities of the Strengthening Families Program and to raise the necessities for its adaptation to Brazil. The results show a consensus among external observers, participants, implementation team, and management team regarding the need to perform adaptations in the superficial structure of SFP, including adaptations in the materials, activities, and language. Resembling findings of studies about SFP’s cultural adaptation carried out in other countries . As such, the cultural adaptation of SFP in Brazil must account for the adults’ illiteracy and the functional illiteracy of younger children. As discussed by Burkhart , but with the participation of all the involved in the implementation process, along with key informants of the target community and members of the management of the service to adopt the intervention (Ferrer-Wreder et al., In summary, this study provided evidence that SFP was perceived as sufficiently appealing, culturally relevant, and partially clear. Efforts for cultural adaptation should be targeted on linguistic aspects of the materials and procedures, considering the cultural, economical, and educational differences of the participant families, and carried on by culturally competent committees in a participatory way. Focus on implementation quality, including infrastructure, families’ mobilization, and continuous planning, should be prioritized. Lastly, a better understanding is needed for the impact of political context on the implementation process and the cultural adequacy of SFP for Brazilian families from non-urban areas, taking into account the social inequalities among Brazil’s regions and its geographic heterogeneity.


Allivibrio fischeri were systematically investigated by using a comprehensive approach including the co-toxicity coefficient (CTC) integrated with confidence interval (CI) method (CTCICI), concentration-response curve (CRC), and isobole analysis. The direct equipartition ray (EquRay) design was used to design five binary mixtures of enantiomers according to molar ratios of 1:5, 2:4, 3:3, 4:2, and 5:1. The toxicities of chiral ILs and their mixtures were determined using the microplate toxicity analysis (MTA) method. Concentration addition (CA) and independent action (IA) were used as the additive reference models to construct the predicted CRC and isobole of mixtures. On the whole, there was an enantioselective toxicity difference between [BMIM]D-Lac and [BMIM]L-Lac, and [HMIM]D-Lac and [HMIM]L-Lac, while no enantioselective toxicity difference was observed for [EMIM]D-Lac and [EMIM]L-Lac, and [OMIM]D-Lac and [OMIM]L-Lac. Thereinto, the enantiomer mixtures of [BMIM]D-Lac and [BMIM]L-Lac, and [HMIM]D-Lac and [HMIM]L-Lac presented antagonistic action, and the enantiomer mixtures of [EMIM]D-Lac and [EMIM]L-Lac, and [OMIM]D-Lac and [OMIM]L-Lac overall presented additive action. Moreover, the greatest antagonistic toxicity interaction occurred at the equimolar ratio of enantiomers. Based on these results, we proposed two hypotheses, (1) chiral molecules with enantioselective toxicity difference tended to produce toxicity interactions, (2) the highest or lowest toxicity was usually at the equimolar ratio and its adjacent ratio for the enantiomer mixture. These hypotheses will need to be further validated by other enantiomer mixtures. Chirality is an important property of molecules. The study of biological activity and toxicity of chiral molecules has important theoretical and practical significance for toxicology, pharmacology, and environmental science. The toxicological significance of chiral ionic liquids (ILs) has not been well revealed. In the present study, the enantiomeric joint toxicities of four pairs of chiral ILs 1-alkyl-3-methylimidazolium lactate to  Chirality is the geometric characteristic of a molecule being not superimposable on its mirror image formed by inversion through a point by pure rotation and translation. Chiral molecules with a single chiral center have two stereochemical arrangements, which have identical physicochemical properties, but can differ in biological activity and toxicity . BiologiS-dinotefuran was 41.1 to 128.4 fold more toxic than R-dinotefuran to honeybee, whereas R-dinotefuran exhibited comparative insecticidal activities (1.7–2.4 times) compared to the racemic mixtures L-(+)-tartrate treatment was greater than [RMIM]D-(-)-tartrate with enantioselectivity D-Lac and [EMIM]L-Lac, [BMIM]D-Lac and [BMIM]L-Lac, [HMIM]D-Lac and [HMIM]L-Lac, and [OMIM]D-Lac and [OMIM]L-Lac. Their specific information is shown in the section of Materials and Methods. Among them, the imidazole cations were not chiral. While lactate anion has two enantiomers of L-lactate and D-lactate, and the human body can only metabolize the L-lactic acid . We wantAllivibrio fischeri (AVF) in a concentration-dependent manner, with log-sigmoidal CRC for the four pairs of chiral ILs shown in 0) and following diluted solutions (C1–C11) of chiral ILs and their mixtures were shown in R2 > 0.91. The variability of the blank control in the test was controlled within ±20%. The indicators of effect concentration EC80, EC50, and EC30 are shown in 50 of EL was 143 times that of OL, and the EC50 of ED was 137 times that of OD. Stock et al. also reported that ILs with long alkyl chains showed higher AVF inhibitive toxicity D-Lac), 1-ethyl-3-methylimidazolium L-lactate ([EMIM]L-Lac), 1-butyl-3-methylimidazolium D-lactate ([BMIM]D-Lac), 1-butyl-3-methylimidazolium L-lactate ([BMIM]L-Lac), 1-hexyl-3-methylimidazolium D-lactate ([HMIM]D-Lac), 1-hexyl-3-methylimidazolium L-lactate ([HMIM]L-Lac), 1-octyl-3-methylimidazolium D-lactate ([OMIM]D-Lac), and 1-octyl-3-methylimidazolium L-lactate ([OMIM]L-Lac). These ILs were purchased from Shanghai Chengjie Chemical Co. LTD. . The chemical structures and related information of these ILs are shown in 0) and following diluted solutions (C1–C11) of chiral ILs and their mixtures were shown in the The stock solutions of ILs were separately prepared through dissolving them in the deionized water and stored in a 4 °C refrigerator. The stock solutions of IL mixtures were prepared through mixing the stock solutions of individual ILs according to their concentration ratios assigned. The concentration of the stock solutions (CAliivibrio fischeri (Strain number 1H00019) was purchased from Marine Culture Collection of China (MCCC). The culture medium consisted of 1 g KH2PO4, 4.7 g Na2HPO4·12H2O, 0.3 g MgSO4·7H2O, 0.5 g (NH4)2HPO4, 30 g NaCl, 5.0 g yeast extract powder, 5.0 g tryptone, 3.0 g glycerin, and 1000 mL water, and was adjusted to pH 6.7 ± 0.3. The AVF was grown in the culture medium at 22 ± 1 °C by shaking (120 r/min) for 8–12 h during the logarithmic growth phase until the relative light unit reached 1 × 106 for the toxicity test.The photobacterium The toxicities of single ILs and their mixtures were expressed as an inhibition of the AVF luminescence. According to the methods of MTA , IL chemE of x%) of individual ILs and their mixtures was calculated using Equation (1). The CRCs were fitted by Weibull function shown in Equation (2) using least squares method [R2 and RMSE. As a quantitative measure of the uncertainty, the observation-based 95% CI was determined [L0 is the average of RLUs of controls, L is the average of RLUs of treatments, E is inhibitive effect of AVF luminescence, C is chemical concentration, a is location parameter, and b is slope parameter.The inhibitive effect (s method . The gootermined .(1)E=1−The five binary mixtures of each pair of enantiomers were designed using the EquRay design , every tx,mix) corresponding to the mixture x% effect, and the predicted CRC of the mixture was also presented [x,mix was multiplied by the enantiomer concentration fraction (iP) to obtain the two partial concentrations which formed a point in the two-dimensional Cartesian coordinates. These points were connected to form the mixture predicted isobole. When the CIs of mixture observed isobole were containing, above, or below the mixture predicted isobole, the mixture was judged to present additive, antagonistic, or synergistic action, respectively.The models of CA shown in Equation (3) and IA shown in Equation (4) were used to predict the mixture effect concentration  to


The authors conducted a prospective longitudinal study from 2009 to 2016 to assess the short and long-term impact of a formal mentorship program on junior faculty satisfaction and productivity. Junior faculty mentees enrolled in the program and junior faculty without formal mentorship were administered surveys before and after the program to assess satisfaction with their mentoring experiences. Long-term retention, promotion, and funding data were also collected. Twenty-three junior faculty mentees and 91 junior faculty controls were included in the study. Mentees came from the Departments of Radiation Oncology and Anesthesia, Critical Care, and Pain Management. After participating in the mentorship program, mentees demonstrated an increase in satisfaction from baseline in five of seven domains related to mentoring, while controls experienced no significant change in satisfaction in six of the seven domains. At long-term follow up, mentees were more likely than controls to hold senior faculty positions  despite no difference in initial administrative rank. When comparing the subset of faculty who were Instructors at baseline, mentees were more likely to be funded and/or promoted than controls (p = 0.030). A majority of mentees reported that the program strengthened their long-term success, and many maintained their original mentoring relationships and formed new ones, highlighting the strong culture of mentorship that was instilled. Several short-term and long-term benefits were fostered from this formal mentorship program. These findings highlight the potential impact of mentorship programs in propagating a culture of mentorship and excellence. Mentorship often assumes an integral role in the mission of academic medical centers because it is considered synonymous with the growth and development of junior faculty. Not simply counseling or teaching, mentoring is a complex interaction grounded by a close relationship between a junior mentee and a senior mentor . Many reA successful mentoring relationship is well poised to fulfill several needs of junior faculty. Mentees often express interest in receiving career development support from mentors, including building a professional network, publishing research, and applying for grants . Some stConsidering the many competing demands of mentors and medical centers, precision about the short-term and long-term effects of a formal mentoring program is especially important to guide the allocation of resources to these programs and ensure that cost is proportional to benefit –14. SpecIn 2009, a survey to faculty revealed interest in exploring formal mentorship, and with support of senior leadership, a pilot formal mentorship program was established at Massachusetts General Hospital (MGH) via the Center for Faculty Development (CFD). The CFD worked with an external consultant and representatives from the Departments of Radiation Oncology and Anesthesia, Critical Care and Pain Management to tailor a formal mentorship program. Junior and senior faculty members in the two departments were invited to participate. Mentor pairs were established by asking junior faculty mentees to rank the five areas of professional development most important to them and by asking senior faculty mentors to rank the five areas that they were best able to provide mentorship in, and then matching the most compatible faculty. Senior faculty members familiar with the mentees and mentors assisted with directing the pairings and securing mentor involvement. The control group for this study was comprised of junior faculty not in the Departments of Radiation Oncology or Anesthesia, Critical Care, and Pain Management and who completed the first questionnaire that was sent to all junior faculty at MGH. Institutional review board approval by the Partners Human Research Committee was obtained for this study and for obtaining informed consent and authorization from participants by voluntary completion of questionnaires.The pilot formal mentorship program consisted of assignment of mentor pairs, three formal training sessions over the course of nine months, and regular informal meetings throughout the program. At the first two-hour training session, mentor pairs received an introduction to mentoring and formalized their action plan. Specifically, MGH leadership introduced the mission of the program, participants discussed best practices in a mentoring relationship, and mentor pairs created an action plan for their relationship that detailed joint expectations, guidelines and boundaries, strategies for addressing stumbling blocks, and goals. At the second two-hour training session, halfway through the pilot formal mentorship program, mentor pairs were given the opportunity to recalibrate their relationship and revisit their action plan. Participants discussed additional concepts of mentoring, such as emotional intelligence and phases of mentoring, worked through case scenarios of challenging mentoring situations, and mentor pairs discussed their relationships and revisited their commitments and goals. The final training session was a closing point for the formal mentoring relationships and served as an opportunity to set the stage for the transition to informal mentoring relationships. Participants discussed how to redefine their mentoring relationship, shared with other participants their best practices and plans for the future, and mentor pairs reflected on their mentoring relationship, on what was accomplished, and how they might work together moving forward. Mentor pairs were encouraged to have frequent informal meetings throughout the duration of the formal mentorship program as well as after its conclusion.Faculty completed three questionnaires over the course of this study. The first questionnaire  Files waThe first and second questionnaires were divided into seven blocks of questions related to a specific aspect of mentoring. These blocks were “desire to be mentored,” “satisfaction with mentoring,” “personal achievement,” “satisfaction with work environment,” “challenges to mentoring,” “importance of attributes of mentor,” and “department fairness and cohesiveness.” The scale of responses to the questionnaire was a 5-point ordinal scale with two additional categories of “not applicable” and “too early to tell.” Depending on the nature of a particular question, the ordinal scale ranged from 1, indicating “not important, not satisfied or strongly disagree” to 5, indicating “extremely important, extremely satisfied or strongly agree.” The questionnaires were constructed by faculty specialized in faculty development by closely adapting common questionnaire format and content to generate questionnaires specific to our novel intervention, and thus were not previously validated questionnaires. The third questionnaire was administered to the faculty to assess subjective long-term impact of the pilot program and their current experiences as a mentee and a mentor, if applicable, in eight domains applicable to mentoring. Depending on the nature of each question, respondents either described the impact of the program on each domain by choosing a response from a 5-point ordinal scale and an additional category “not applicable,” or chose any domains that applied to a particular question. Respondents were also allowed to provide open-ended responses to describe why they did not continue an old mentoring relationship or establish a new mentoring relationship.Long-term assessment of the effects of the mentorship program was comprised of subjective results from the third questionnaire, as well as objective data collected seven years after the beginning of the formal mentorship program. Retention rate data, former and current academic rank of participants, and grant funding information of the mentees participating in the mentoring program and the control junior faculty who remained at MGH were collected.The block of questions in the first and second questionnaire contained between 7 and 21 individual sub-categories, and the sum of responses for each response score was tabulated to give an overall weight of that response score in the block. Response scores were also analyzed by placing them into two groups, with the “unsatisfactory” group containing response scores 1–3 and the “satisfactory” group containing response scores 4 and 5. The division of response scores into two groups better enabled tracking changes in responses between the two groups from before the mentoring program to after the program as well as between the study and control cohorts. The satisfactory and unsatisfactory groups were converted to proportions, and a Wald 95% confidence interval was generated for each proportion. To determine whether the median response within each group of questions was the same at baseline as after the program, we used a nonparametric Wilcoxon-Mann-Whitney U test. We used the Kolmogorov-Smirnov two-sample test to check the validity of the assumption that the distributions of responses at baseline and after the program were the same. Exact p values were calculated for both tests. In addition, for each group of questions, we tested the hypothesis that the proportion of satisfactory group responses was higher after the program than it was at baseline using a one-sided exact test for binomial random variable to test improved responses. All other tests were two sided. P < 0.05 was deemed statistically significant for all tests. Odds ratios (OR) and 95% confidence intervals were calculated to determine the likelihood of an improved score in the post-mentoring group.The response scores from the third questionnaire were also placed into unsatisfactory and satisfactory groups. Because of the small sample size, only a descriptive summary of the results from the subjective third questionnaire is provided. The mentee and control cohort were compared using the Fisher’s exact test for categorical variables and the Wilcoxon rank-sum test for continuous variables. The independence of categorical variables was evaluated with the Pearson’s chi-squared test. Mentees were defined as holding junior ranking if they were a Lecturer, Instructor, or Assistant Professor, and senior ranking if they were an Associate Professor or Full Professor. To holistically assess any impact the mentoring program may have had on overall success in academic medicine, the metric “funded + promoted” was devised to sum the number of times a faculty member was promoted since the pilot mentorship program with either 1 or 0 if the faculty member had or had not received any funding, respectively.The 23 mentees that participated in the pilot mentoring program were contacted and administered the first survey before the start of the mentoring program. Three mentees subsequently left the mentoring program, and of the 20 remaining mentees, 14 (70%) responded to the second questionnaire sent at the end of the mentoring program. Of the 91 junior faculty members not in the mentoring program that completed the first questionnaire that was administered before the start of the mentoring program, a random sample of 50 was sent the second questionnaire after the end of the mentoring program, and 40 responded (80%).Within the mentee group, satisfaction after the mentoring program in blocks 1–5 was significantly improved from baseline . Ten perAt baseline, we found that the distribution of responses and the median responses of the mentee and control groups were not statistically different except for in blocks 2 and 3, the domains describing satisfaction with mentoring and personal achievement. In these blocks, mentees demonstrated a lower satisfaction with mentoring (p = 0.01) and personal achievement (p = 0.001) than the control group. One year after the start of the mentoring program, there was no statistically significant difference between the mentee group and the control group across all question blocks. For blocks 2 and 3, the mentee group started at baseline with a lower proportion of “satisfactory” answers than the control group, but ended up with a similar proportion of “satisfactory” answers compared to the control group after the program was completed.2  = 2.2488, p = 0.134). Mentees received a median external funding amount of $1,783,200 and controls received a median of $495,900 in external funding, but this difference did not reach statistical significance (p = 0.21). 2  = 8.9514, p = 0.030, with mentees being more likely to be funded and/or promoted than controls. When examined separately, neither promotion nor funding reached a significant association with group status.Of the 40 responding control faculty, only 32 provided identification, which was necessary for long-term follow up. The institution retention rate in 2016 for mentees was 15/20 (75%) and for controls was 23/32 (72%). Only the faculty that remained at MGH were assessed for long-term academic promotion and grant funding amounts. Mentees were more likely to hold senior faculty positions at long-term follow up (p = 0.030) despite no difference in initial rank (p = 0.22) or other baseline characteristics . SpecifiOf the 15 mentees remaining at MGH, 11 (73%) responded to the third long-term subjective questionnaire in 2016. A majority of respondents stated that they thought that the pilot mentorship program in 2009 had strengthened their ability to achieve academic promotion (7/11), access a larger professional network and obtain increased professional visibility (7/11), obtain more leadership positions (6/11), and serve as a teacher and mentor (6/11) . Five meRecent reports have demonstrated tangible benefits of a mentorship program on faculty development. Our study builds on these recent reports by examining several domains of mentorship. Unlike many previous reports, our study consisted of a prospective long-term comparison of mentees and a control population in both subjective and objective areas, and allows for novel observations. We found that faculty who participated in the formal mentorship program derived both short-term and long-term benefits in comparison to faculty who did not. Mentees reported subjective improvements in their productivity and satisfaction in many areas, and demonstrated higher objective achievement than controls.To assess the validity of comparing the faculty who participated in the pilot mentoring program with the control faculty, we evaluated baseline perceptions of mentoring and academic rank for both groups. We found that there was no statistical difference in desire to be mentored, satisfaction with their work environment, perceptions of challenges in being mentored, ascribed importance of mentor attributes, nor perception of department fairness and cohesiveness. Finally, there was no statistical difference in baseline academic rank of mentees compared to control faculty. Thus, given the several baseline similarities of the mentee and control faculty with regards to their perceptions of their department, their expectations of mentoring, and their objective achievement, we found it appropriate to compare the short-term and long-term differences between the two groups.We found that upon concluding the pilot mentorship program, mentee attitudes changed significantly from baseline. Among other changes, mentee satisfaction with level of personal achievement increased from 29% to 50%, and their satisfaction with their work environment increased from 35% to 65%. While these are significant improvements, there is still room for improvement in the overall level of satisfaction in each domain. The control group, on the other hand, experienced no significant changes in any domains from the beginning of the mentoring program to the end other than a 9% increase in satisfaction with their work environment. The positive externalities of the pilot mentorship program appear to persist for several years, and these same domains remain a focus for the mentees. A majority of mentees still work with mentors to increase their ability to achieve promotion, leadership positions, and a broader professional network, and most mentees cite participation in the pilot mentorship program as integral to their improvement in these domains. The subjective intuition of the mentees is supported by long-term follow up data; despite no difference in baseline academic rank, by 2016, mentees were more likely to hold senior rankings than controls. Mentees received over triple the amount of median external funding than controls, and while this difference did not meet statistical significance, it trends towards importance. By adjusting for initial ranking by only comparing faculty who were Instructors at baseline, mentees were found to be more likely to be funded and/or promoted than controls. While separately there was no difference between groups in receiving a promotion or in receiving funding, pooling these two important indicators of success in this relatively small cohort may allow for the detection of the holistic effects of a mentoring program on overall success in academic medicine. Several years after the conclusion of the pilot mentoring program, five of 11 responding mentees continued their original mentoring relationship, and nine secured additional mentors to assist them with their evolving goals. Importantly, the vast majority of mentees became mentors themselves to a combined 57 mentees, highlighting the strong culture of mentorship that was instilled.Mentorship is widely regarded as important to the fostering of young faculty at centers of academic medicine. However, these institutions and the senior faculty who would serve as mentors have limited time and funding, and thus mentoring may be underutilized without clear evidence of its benefits ,14. HereOur study has some limitations. Mentees came from only two departments, the Department of Radiation Oncology and the Department of Anesthesia, Critical Care and Pain Management. Holliday et al. examined mentorship in radiation oncology, and highlighted that the improvements seen in their study may be attributed to the fact that the competitiveness of radiation oncology necessitates that faculty utilize any available resource, such as mentoring, to not fall behind their peers . FurtherIn summary, we report several benefits derived from a reproducible intervention that was aimed at improving the quality of mentoring relationships. This study provides evidence to support the adoption and expansion of mentoring programs at other institutions to help propagate a culture of mentorship and excellence.S1 FileThe first questionnaire sent to mentees before the formal mentorship program began.(PDF)Click here for additional data file.S2 FileThe first questionnaire sent to controls before the formal mentorship program began.(PDF)Click here for additional data file.S3 FileThe second questionnaire sent to mentees after the formal mentorship program ended.(PDF)Click here for additional data file.S4 FileThe second questionnaire sent to controls after the formal mentorship program ended.(PDF)Click here for additional data file.S5 FileThe third questionnaire sent to mentees several years after the formal mentorship program ended.(PDF)Click here for additional data file.


Intervention fidelity was assessed from 1964 video recordings of intervention sessions submitted for certification evaluation using the observation-based measure Fidelity of Implementation Rating System (FIMP). A small but significant drop in fidelity scores was previously observed from G1 to G2; however, fidelity scores recovered at or above G1 scores for G3 Forgatch and DeGarmo  and the Norwegian team trained the next six generations (G2–G7). The mean rate of certification was 94.2% ( The era of developing and testing parent training programs that address children’s behavior problems has yielded a rich crop of empirically supported interventions , a national center for implementation and research that has been the stable base for the nationwide implementation of GenerationPMTO for nearly two decades. The full transfer approach to implementation grew out of this collaboration.In full transfer, the program developer trains a first generation (G1) of practitioners to certification and gradually turns over program authority to the adopting community. From the G1 certified GenerationPMTO therapists, leaders were selected to form an infrastructure to carry the program forward to future generations of practitioners, trainers, coaches, and fidelity raters . The NIT serves as the kingpin for the implementation in Norway, acting as the executive governing authority for PMTO. When the developer rebranded to GenerationPMTO, the Norwegian program retained the original nomenclature, PMTO. NCCBD made a long-term strategic plan to conduct future implementation activities in Norway. These activities include the training, coaching, evaluation of intervention fidelity, monitoring of practice, expansion of subsequent generations of PMTO practitioners, and adaptations for specific contexts and populations.Findings for the first three generations of therapists trained in this full-transfer approach were reported in Forgatch and DeGarmo . The autImplementation research examines processes and strategies involved in the installation of evidence-based interventions in usual care settings. Proctor et al.  categoriThe aim of the Norwegian implementation has been to install and extend the nationwide reach of the PMTO model to diverse agencies and families seeking help while sustaining intervention fidelity and positive family outcomes. Successful outcomes require recruiting, training, coaching, and sustaining substantial numbers of certified practitioners who practice the program with high fidelity to the intervention. We present data across seven generations of Norwegian practitioners who initiated training in the PMTO model with data from 1964 observations of 491 PMTO therapists across diverse service systems throughout the nation.Adoption: Total numbers of therapists and participating agencies increase with each generation, and percentages of therapists who complete training with certification are at or above the G1 level.Sustainability: Therapists continue to practice PMTO with certification and agencies continue their participation.Reach: The shift in focus from clinical to preventive services called for less educational requirements of the practitioners and broadening types of community service agencies. Thus, we hypothesize higher percentages of therapists with less educational level and greater participation of organizations providing primary service in the municipalities.Fidelity: Fidelity is sustained at or above G1 levels across seven generations with the exception of the previously reported dip for G2. In the earlier report, we anticipated continuing decline in fidelity following the transfer. Given the recovery for G3, we now hypothesize that fidelity will be sustained through G7.We test the following hypotheses:The transfer of PMTO from the program developer to Norway began as the developer’s team initiated training the G1 clinicians in 1999 .Participants were professionals who initiated a PMTO training program between 1999 and 2017. Data are based on these seven generations of trainees .decision to implement a program  and active teach .Process: Promotes a safe and supportive learning context.Overall Development: Promotes growth in PMTO use, therapeutic relationship, apparent satisfaction, managing difficult contexts/issues.Fidelity has been defined as the degree to which an intervention is implemented with adherence and delivery quality as intended by the program developers , 4–6 is “acceptable,” and 7–9 is “good work.” We present the Total FIMP Score, which is the mean of the sum of the scores from the five categories on a given session.Studies have evaluated the predictive validity of FIMP with the hypothesis that high levels of fidelity scored from session observations account for significant variance in pre/post-change in parenting practices, the putative mediator of child outcomes, as well as change in distal child behavior outcomes. The first validity test of FIMP was based on an efficacy trial with an at-risk prevention sample of stepfamilies  score of 70% or higher is required. The model developers trained the leaders of the Norwegian FIMP team in 2003. The Norwegian FIMP team has trained later generations of Norwegian FIMP coders.Members of the Norwegian FIMP team were selected within the NIT, based on professional competence, high FIMP scores on their work, and personal skills. For the first 10 years, most FIMP coders were psychologists. Thereafter, the team included experienced therapists, educators, and social workers with comprehensive PMTO experience.FIMP coders independently score and discuss videotaped examples in regular meetings during each certification period and at the NIT meetings. FIMP coders are also required to reach criteria in an annual international reliability test administered by the developer. Coders who do not reach criteria are given makeup tests.encouragement and discipline. The criterion for achieving certification is a score of 6.0 or higher on the mean Total FIMP Score, which encompasses all five categories for each of the four certification sessions. From G2 onward, the Norwegian FIMP team has had the main responsibility for certification. To prevent bias, FIMP coders do not score certification sessions from therapists in their region. When there is uncertainty about pass or fail certification status, the FIMP team determines the outcome in consensus discussion. In case of disagreement within the team, the two FIMP leaders jointly decide the final scores. If a session tape receives a failing score, the trainee must submit another session tape.Each trainee submits four videotapes from sessions introducing and troubleshooting Table Over two decades, 491 of 521 (94.2%) therapists completed the extensive training program with certification. The percentage of completers by generation ranges from a low of 85% for G1 trained by the developer’s team to a high of 100% trained by Norwegian teams. The mean percentage rate of completion when practitioners were trained by the Norwegian team was 95.8%. These data show program adoption to grow with each generation with high numbers of therapists participating and substantial numbers completing the training with certification.n = 19) included change of job, moving, illness, and other personal issues. Trainees with failing certification sessions (n = 75) are given three opportunities to resubmit. They receive comprehensive written feedback describing what must be changed and are offered extra coaching. Of the 502 candidates who submitted tapes for certification, only 2.2% (n = 11) never achieved a passing score. Number of agencies providing services increased from 19 agencies in G1 to 164 active agencies through G7.Of those who began training, 30 (5.8%) did not certify. Reasons for not completing . From G5–G7, 61.6 to 85.7% were recruited from the primary services in the municipalities. Diversity of agencies from G1 to G7 increased from 24 to 112 PMTO practitioners employed in mental health out-patient clinics, from 10 to 139 in child welfare agencies, and from zero to 240 in primary service agencies. During this period, approximately 22,000 families received PMTO in mental health, child welfare and 78 of 422 municipalities from all regions in Norway.Reach hypotheses take into account the longer-term implementation goal to shift from a clinical to a more preventive focus. This led to the following: (1) practitioner educational levels will decline and (2) service agency diversity will increase and shift from national to municipality levels. Data support the hypotheses:This hypothesis tests the extent to which high intervention fidelity is sustained beyond the three generations previously studied. As Table The aim of the Norwegian implementation was to adopt the PMTO program and assume executive authority to install the program nationwide, with the ultimate goal of extending its reach to families seeking help within clinical and prevention services throughout the nation. To accomplish this, the implementation had to address multiple challenges. First, PMTO would have to be palatable to Norwegian service systems, practitioners, and families so that they would adopt the program. Sufficient numbers of practitioners would have to be trained to certification with high levels of intervention fidelity. Reach would have to be extended to cover a range of services from clinical to preventive populations throughout the nation. Then, it would be necessary to sustain the quality of training and coaching for multiple generations of therapists. To maintain growth and continued practice with intervention fidelity, an implementation infrastructure would have to be established and maintained. Finally, practitioners would have to be motivated to sustain certification to provide services with intervention fidelity.Important success factors in achieving these goals included building and maintaining high levels of intervention and implementation competence among each PMTO generation. G1, who would have an essential role in the scale-up, were recruited mainly from mental health and child welfare clinical services. The most qualified were selected to become members of the NIT; others were chosen to establish clusters of trainers and coaches for subsequent generations within their sites. From G3 onward, recruitment was from primary services in the municipal communities, predominantly working at the selected and indicated levels of prevention. From each generation, the NIT has selected the most competent to become trainers and coaches for following generations. This approach to building capacity and increasing the implementation infrastructure has advanced the scale-up.The NIT has integrated PMTO into key systems serving families: clinical services at the national level within child mental health and child welfare, and a variety of primary services in the municipalities. The stability of the NIT, which consisted of the same persons during the first 10 years, has been an important success factor. Today, it consists of 20 highly competent and committed professionals. The NIT has provided a powerful foundation for the nationwide implementation activities, including the establishment of training and certification protocol for trainers, coaches, and practitioners.The Norwegian implementation has produced widespread and long-lasting adoption of PMTO. We assessed uptake in terms of growing numbers of certified practitioners and agencies. From 34 G1 practitioners certified in 2001, the numbers grew to 491 certified by 2018, of these 314 are still active. Certified therapists maintained their certification and continued practicing over years. For example, of those trained in G1 two decades ago, 24% maintained their certification, and practitioners from subsequent generations have kept up their certification with excellent retention rates. An important implementation strategy included recruiting clusters of certified therapists to serve as trainers and coaches in each geographical area to enhance sustainability and promote quality service  led the NIT to clarify qualifications and personal characteristics that are required in the recruitment of new generations. Over the years, NIT has specified several selection characteristics and become more active in the recruitment process.Obtaining and maintaining commitment and support from agency leaders continues as a challenge. It has helped to require that each service agency provide written agreements that enable their candidates to fulfill the demands of the training program and participate in the long-term implementation process. For instance, the top administrative directors and executive leaders agree to permit the trainees to spend a minimum of 40% of their working time participating in training activities and offering PMTO services to at least five families per candidate.Mandatory participation in coaching groups presented a challenge. During the early generations, agency leaders claimed that coaching was too time consuming, especially for trainees who had to travel far. NCCBD addressed this problem by recruiting more candidates from the same geographic area and providing local coaching groups. Regular local coaching for trainees throughout their 18-month training period has promoted high rates of certified therapists and high levels of fidelity. Practitioners are supported by local coaches and trainers, who are supported by the NIT, a process that contributes to the maintenance of certified therapists. Strong local networks, as well as regular regional and national boosters, provide local practitioners with a sense of belonging to a larger community. All share the mission of offering effective and evidence-based intervention for children and families.Maintaining motivation at every level in the PMTO system is a challenge that is addressed by the NIT’s attention to nationwide connectivity. Practitioners are also inspired by findings from the studies conducted by the NCCBD research team with positive outcomes for children and families and successful adaptations of the program (e.g., Ogden and Hagen One limitation in this study is that fidelity was assessed only at certification. Regular fidelity assessment should take place to ensure sustained intervention fidelity (Sigmarsdóttir et al. The Norwegian program marked the first time the developers of GenerationPMTO implemented the intervention beyond their own team. Although they had conducted multiple efficacy trials at clinical and prevention levels (see Dishion et al. The Norwegian program has benefitted from strong and reliable funding provided to a coordinating center, NCCBD. In turn, NCCBD has served as the home for implementation and research activities that evaluate processes and outcomes, as well as provided for further development, adaptation, and quality assurance. Some may claim that the certainty of stable financial and political guarantee in the form of a fixed item on the national budget has ensured the success of the implementation. Indeed, this has facilitated the quality of work and the capacity to conduct extensive research. However, a favorable financial situation is not enough to achieve goals that enhance adoption, such as training programs that result in large numbers of practitioners who achieve certification and continue their practice in the model with high fidelity. Without such a growing force of committed and competent practitioners, extending the program’s nationwide reach across multiple service sources would be unlikely.The sustainability of PMTO in Norway over nearly two decades rests on a strong infrastructure, carefully articulated implementation strategy, collaborative relationship between the developer and the adopter, continuous economic support from governmental grants, and careful attention to intervention fidelity for each new generation of therapists. Over the years, the core principles and procedures of the PMTO program have been maintained without substantial changes. Norway has installed and maintained effective PMTO in environments dedicated to adopt and nurture the model and the practitioners.


Determining whether a professional development program can be enacted with integrity in different settings and by different facilitators is critical to understanding efficacy. In this paper, we describe the two-stage preparation process of a facilitator as she prepared to use and adapt the highly specified Learning and Teaching Geometry video-based professional development materials with fidelity. The latter stage of the preparation process involved a rehearsal, during which the research team used two instruments to measure fidelity.Two existing instruments were used to explore fidelity through different lenses, including timing and modification of activities and learning goals.Results from both fidelity instruments indicate that the facilitator used the materials as intended by the developers. However, these instruments did not capture important information regarding modifications the facilitator made, including timing and content-focused adaptations.Suggestions are made with respect to measuring fidelity, preparing facilitators, and supporting productive adaptations. Professional development (PD) for mathematics teachers is central to efforts to improve classroom instruction and student learning, both in the USA and internationally. Building on classroom research that highlights the interaction between the curricular materials, teachers, students, and the local context  video-based mathematics professional development  and contain critical information needed to lead a given session. Each agenda lists the main mathematical focuses  of the session, and for every activity in the session, the agenda includes a detailed description, a suggested time allotment, the necessary materials, guiding questions, and extensive notes . Having session agendas with predictable structures is intended to support facilitators in using the materials with integrity to the LTG PD goals and principles while also shaping the parameters for both adherence and flexibility.Preliminary research on the effectiveness of the LTG PD program, which took place concurrent with its development, offers evidence of the promise of the program to impact teacher and student learning. A portion of the LTG PD materials, the Foundation Module, was piloted in eight sites throughout the USA in order to generate both formative and summative evaluation data. Based on data collected from this pilot, the LTG PD program was shown to lead to significant gains in teachers’ geometry content knowledge, along with the knowledge to effectively convey that information in the classroom. On a content knowledge assessment, the treatment teachers demonstrated an average gain of almost 9 percentage points from pretest to posttest, which was significantly higher than the comparison teachers’ average gain of less than 2 percentage points. Similarly, on assessments embedded within the PD that addressed content and pedagogical content knowledge, the treatment teachers significantly improved on five of the six questions, whereas the comparison teachers did not show significant improvement on any question  and their students. Approximately half of the teachers were randomly assigned to take part in the LTG PD in the first intervention year and half will take part in the second intervention year. The intervention consists of the entire LTG PD program, including a 1-week summer institute and 4 days of academic year follow-up sessions beginning in Summer 2016.As a phase two research endeavor, a primary goal of the LTG Efficacy Study’s central goal is to determine whether the PD program can be enacted with integrity in various settings by a facilitator who was not a developer of the materials Borko . Most coIn efficacy studies, it is essential to demonstrate that a facilitator can implement the PD program with fidelity, in order to argue that any demonstrated impact (or lack thereof) is a true reflection of the PD program and cannot be attributed to implementation failure  by going through all of the provided facilitation resources, working out the mathematics problems, watching the associated video clips, and thinking carefully about recommended facilitation strategies. She kept written notes about her ideas and questions and shared those notes with the co-PIs prior to our meetings. The co-PIs audio taped and took notes during each meeting, documenting the manner in which Hannah’s concerns and questions were taken up and addressed.A central component of Hannah’s individual study was her attention to the specifics in each session agenda. Hannah was highly attentive to the nature of each workshop activity, including the timing and purpose of the activity, the mathematical goals, and the suggested probing questions. Hannah made notes for herself in the margins of the agendas, which she later drew on when facilitating. These notes included topics that she wanted to be sure to give enough attention to, questions or concerns she had about how teachers would respond to an activity or probe, and additional questions to ask or mathematical connections to make if appropriate. Hannah added to or revised her notes based on conversations with the co-PIs. Prior to the facilitation rehearsal, Hannah again independently reviewed the session materials and her notes.In February 2016, Hannah conducted a 2.5-day rehearsal of the first half of the LTG Foundation Module in order to practice using the materials and to determine whether her facilitation was in line with the developers’ intentions. Hannah drew upon her Stage I work and knowledge of the participants to plan and prepare for the rehearsal. The rehearsal was held at the University of Hawaii in Honolulu with eight participants—including five secondary mathematics teachers from the University Laboratory School in Honolulu, HI, and three mathematics educators from the University of Hawaii at Manoa College of Education’s Curriculum Research and Development Group. As a recently retired faculty member of the University of Hawaii and former secondary math teacher at the University Laboratory School, Hannah knew all of the participants. The three co-PIs attended and videotaped the rehearsal. In addition, the co-PIs documented, individually rated, and came to consensus agreement on Hannah’s fidelity of implementation ratingsFidelity of the rehearsal workshops was analyzed using modified versions of two existing instruments in order to explore fidelity through different lenses, based on the current literature. Both instruments are session-specific, meaning that they take into account fidelity to the written agenda for each workshop session. As such, both instruments had to be modified to incorporate the specific activities or stated goals for each session of the LTG PD program. Together, the instruments capture adherence to the intended mathematical and pedagogical storylines and the degree to which modifications were made.The first fidelity instrument was created by Horizon Research, Inc. (HRI) specifically for the LTG PD program and was slightly adapted for this study to account for an updated version of the PD materials. This instrument, the LTG PD Session Logging Tool, measures (1) adherence to suggested time spent on each activity categoryAs shown in Table The extent to which individual activities were modified during the rehearsal was rated on a 5-point scale using the Session Logging Tool, from not modified at all to substantially modified. However, in Table The second instrument used to measure fidelity of implementation (FOI) of Hannah’s rehearsal was a modified version of the Teacher Learning Goals instrument. The Teacher Learning Goals instrument is derived from a fidelity instrument created by the Center for Elementary Mathematics and Science Education at the University of Chicago , activity categories were left out of the session. The activity categories that were left out of the rehearsal focused on using the PD materials in the teachers’ classrooms , completing a set of mathematics tasks , and reflecting on the last session (instead they reflected on the PD as a whole).The Session Logging Tool captures timing as a component of fidelity without taking into account the appropriateness of any time adjustments. This type of fidelity measurement does not capture the degree to which modifications were made “with integrity” to the designers’ intentions, only to the degree to which they were made. In all cases, the decisions that Hannah made to spend more or less time on a given activity category, or to leave out an activity category, were informally discussed by the raters and deemed to be consonant with the designers’ intentions. Additionally, the instrument does not account for activities that a facilitator may add to a given session. For example, in session 5, Hannah elected to include an activity from a later session in the LTG PD materials since her group would not be doing that session and she felt the activity would be useful for their learning.Both the Session Logging Tool and the Teacher Learning Goals instrument examine the facilitator’s modifications within a session. However, the Session Logging Tool focuses on smaller units (activities), whereas the Teacher Learning Goals instrument focuses on more broadly defined goals. Additionally, the Session Logging Tool captures the degree to which there are modifications within an activity, whereas the Teacher Learning Goals instrument simultaneously captures modifications and their appropriateness. Using both of these instruments, Hannah’s rehearsal of the LTG PD program can be characterized as high fidelity, with appropriate adaptations made as needed based on the PD context. However, neither instrument probed deeply into the specific nature of Hannah’s adaptations, prompting us to qualitatively take up such an endeavor as discussed in the next section.As part of a thorough examination of fidelity of implementation, it is helpful to consider examples of how and why facilitators modify aspects of the target program to fit the needs of the participants and their unique context. Exploring these adaptations helps to determine whether they should be classified as productive, no impact, or fatal. In the case of Hannah’s rehearsal workshops, we categorized her adaptations as falling into two overarching categories: timing and content-focused. Below, we provide specific examples and a broad classification of each type of adaptation.Hannah’s rehearsal was scheduled such that she only had time to go through half of the LTG Foundation Module . Hannah had carefully studied all ten sessions in phase 1 of the preparation process, and because the sessions are intentionally sequenced to follow a specific mathematical trajectory, she knew it was important to go through them in order. Therefore, Hannah planned in advance to use only the first five sessions with her participants. However, Hannah was familiar with the other five sessions and towards the end of the rehearsal she elected to bring in an activity from a later session that fit the participants’ mathematical interests and supported elements of their curriculum.Another timing adaptation Hannah made was to modify the amount of time spent of specific workshop activities. In total, Hannah modified the timing of just under half of the activities in sessions 1 through 5, as shown in Table Hannah’s timing adaptations were judged by the raters to be “no impact” adaptations, using Seago’s  classifiContent-focused adaptations refer to additions or alterations that Hannah made to the PD materials in an effort to meet the perceived mathematical learning needs of her participants. During the rehearsal, the participants voiced an interest in exploring the connection between transformations-based geometry and algebraic concepts, such as slope and graphs of linear equations. Making connections between geometry and algebra is an important feature of the LTG PD program, but it is a feature that does not come into sharp focus until the later sessions of the Foundation Module. After the first day of the rehearsal , Hannah made the decision to leave out a few activity categories in sessions 4 and 5 in order to incorporate some activities from session 8 that she was originally not planning to cover. Specifically, the sections of the materials that Hannah left out were making connections to classroom practice in sessions 4 and 5, and one of the math problems in session 5. All of the activities that Hannah left out were mathematically below the ability level of her participants and pedagogically not particularly relevant to their students. Because the rehearsal included only a few teachers who all taught at the same school, this judgment was clear to Hannah as well as to the fidelity observers, thereby leading the raters to classify it as a “productive adaptation”. The rehearsal teachers were a fairly unique group in the sense that they already had extensive knowledge in the basics of geometric transformations, and several were actively teaching this topic to their students across grade levels. Therefore, moving into more challenging content seemed appropriate given their interest, prior knowledge, and time considerations.Another content-focused adaptation deemed to be “productive” occurred during the discussion of a mathematics task towards the end of session 1 when Hannah introduced the term “inspection method” to conceptually describe one participant’s method for solving the task. Hannah again used this term in session 3 after a teacher explained that he “saw” the solution to a problem based on his basic understanding of mathematical properties and geometric transformations. Later, the term was taken up by other participants, who used it to describe student methods in some of the video clips they watched and to categorize how many students in their own classes work on geometry problems. In their written reflections, several teachers mentioned that they had more appreciation for their students’ visual approaches to math problems. What is interesting about Hannah’s introduction of the term “inspection method” is that the term is not highlighted in the LTG PD materials, nor was it discussed during phase 1 of the preparation period, but it fits readily into the mathematical and pedagogical storyline. Sometimes referred to as “eyeballing,” visual inspection methods rely on carefully studying and reasoning about the visual makeup of an object and are a precursor to more sophisticated mathematical reasoning. By coining a name for this approach, Hannah helped teachers notice its importance in the intended learning trajectory and built on the participants’ ideas to promote multiple mathematical and MKT learning goals.Hannah’s adaptations were clearly in line with the intentions and goals of the materials, and none would be considered “fatal.” She did not venture far from the provided resources and remained focused on the goals within each session to support participants’ learning. Her experiences preparing to facilitate, combined with her growing understanding of the participants’ mathematical knowledge and instructional environment, appeared to support her ability to actively listen and react to the participants’ ideas, seizing upon “teachable moments” while ensuring the integrity of the LTG PD program.Vital to the exploration of a given PD program’s impact is determining whether the program under study is implemented with fidelity. Loucks-Horsley et al.  noted thIn this final section, we consider several issues related to fidelity. First, we explore the connection between fidelity, adaptations, and the improvisational nature of facilitation. Next we consider how fidelity should be measured, particularly in the context of leading a mathematics PD program. Finally, we consider the process of preparing facilitators to lead PD with fidelity and make some conjectures about scaling. Akin to skillful facilitation, our aim is not necessarily to provide answers but rather to bring challenging issues to light for thoughtful consideration.Facilitators, like teachers, must make continual, in-the-moment decisions that take into account the interplay between their PD program, the participants, and the situational context. Whether these decisions impact their fidelity of implementation is a critical issue to consider, for example to understand variation across workshops and to gauge the effectiveness of the PD. Although the LTG PD program is a specified model of PD with a pre-defined set of activities and goals, it rests on the foundation that facilitators will need to make decisions regarding adaptations based on the nature of their participants and the circumstances under which they are working.A number of researchers have argued that classroom teaching can be understood as improvisational performance  have argued, “Given that adaptation happens, a unidimensional view of fidelity of implementation that results in a single score or rating does not accommodate the dynamism of intervention enactment in the real world” (pg. 214). Education researchers interested in measuring fidelity generally report on two broad dimensions: structural and process  is determining which facilitation strategies underlie effective PD workshops and are thus most important for novice facilitators to learn and practice. Facilitation frameworks (such as those provided by Borko et al. Research on preparing and supporting facilitators of mathematics PD is still at a very early stage, but there are meaningful insights to be gained from the more extensive literature on classroom teaching. At the same time, it is important to acknowledge that facilitating a PD program is, in many respects, different from using curricular resources in the classroom. Even PD programs that are highly specified are unlikely to contain as many tools, artifacts, and resources compared to curricula intended for school-aged students. Also, PD programs generally place fewer constraints on facilitators, for example by assuming that adults will play a more central role in shaping their own learning. As such, facilitators are afforded the space to make interpretive moves, which may or may not affect their fidelity of implementation. It is likely that issues involving identity, knowledge, and beliefs will emerge as critical influences on PD facilitators (Gresalfi and Cobb Another interesting question to consider is whether and how facilitator preparation and fidelity differs depending on the nature of the PD program. For example, do facilitators of specified PD programs require more training than facilitators of adaptive PD programs? We conjecture that the facilitation of both types of programs is equally demanding in preparation and practice, but the nature of the demand differs. In adaptive PD programs, facilitators must select activities that support teacher learning opportunities, which is not a demand faced by facilitators of specified PD programs. However, in specified PD programs, facilitators must grapple with understanding someone else’s workshop design and learning goals. With respect to fidelity, documenting adherence to the designers’ intentions and the nature of adaptations is a more straightforward endeavor for specified programs with explicit goals. At the same time, there is certainly a strong need for facilitators of both types of programs to be well versed in the content and pedagogy associated with effective facilitation. The field would benefit from future research studies that examine the similarities and differences between the demands placed on facilitators of adaptive and specified PD programs and ensuring they are prepared to meet those demands.


The prevalence of overweight and obesity in children and adolescents has dramatically increased in the Pacific Island countries and territories over the last decade. Childhood overweight and obesity not only have short-term consequences but are also likely to lead to noncommunicable diseases in adulthood. A major factor contributing to the rising prevalence is an insufficient amount of daily moderate-to-vigorous physical activity (MVPA). In the Pacific region, less than 50% of children and adolescents meet the international recommendations of 11,000 steps and 60 min of MVPA per day. Although studies have shown the potential of digital technologies to change behaviors, none has been proposed to guide adolescents toward achieving these recommendations.The aims of this study were (1) to investigate whether a technology-based educational program that combines education, objective measures of physical activity (PA), and self-assessment of goal achievement would be well received by Pacific adolescents and help change their PA behaviors toward the international PA recommendations and (2) to create more insightful data analysis methods to better understand PA behavior change.A total of 24 adolescents, aged 12 to 14 years, participated in a 4-week program comprising 8 1-hour modules designed to develop health literacy and physical skills. This self-paced user-centered program was delivered via an app and provided health-related learning content as well as goal setting and self-assessment tasks. PA performed during the 4-week program was captured by an activity tracker to support learning and help the adolescents self-assess their achievements against personal goals. The data were analyzed using a consistency rate and daily behavior clustering to reveal any PA changes, particularly regarding adherence to international recommendations.The consistency rate of daily steps revealed that the adolescents reached 11,000 steps per day 48% (approximately 3.4 days per week) of the time in the first week of the program, and this peaked at 59% (approximately 4.1 days per week) toward the end of the program. PA data showed an overall increase during the program, particularly in the less active adolescents, who increased their daily steps by 15% and ultimately reached 11,000 steps more frequently. The consistency of daily behavior clustering showed a 27% increase in adherence to international recommendations in the least active adolescents.Technology-supported educational programs that include self-monitored PA via activity trackers can be successfully delivered to adolescents in schools in remote Pacific areas. New data mining techniques enable innovative analyses of PA engagement based on the international recommendations. The populations of the Pacific Island countries and territories (PICTs) have undergone a rapid lifestyle transition, with impacts on health. Despite a history of colonization, tribal and rural living has persisted, but today’s urbanization, industrialization, and mechanization are causing many transformations , notablyData are limited for the PICTs, but a recent Australian study found that initiatives to increase adolescent MVPA have been unsuccessful . The lowbe fit or be fat), resulting in demotivation and negative feelings.Furthermore, digital interventions aiming to increase PA have shown promising results, particularly when they involve self-monitoring and feedback via activity trackers . HoweverWe explored a technology-supported educational program focused on PA in a Pacific Island rural school environment. Our program  was designed to improve adolescents’ PA knowledge and skills and to help them understand their activity tracker data, which provided objective feedback on their PA. Data mining enabled us to develop a comprehensive approach to analyzing PA trends throughout the program. First, clustering took into account all the components of the international recommendations . Second, a sliding 7-day window captured the frequency and regularity of the PA behaviors.The first aim was to investigate whether a technology-based educational program that combines education, objective measures of PA, and self-assessment of goal achievement would be well received by Pacific adolescents and help change their PA behaviors toward the international PA recommendations. The second aim was to create more insightful data analysis methods to better understand PA behavior change.This exploratory study was conducted in a rural school in New Caledonia on Lifou Island. A total of 24 adolescents aged 12 to 14 years participated in this pilot. Parents gave informed written consent before their child’s participation in the study, which met all legal requirements and the criteria of the Declaration of Helsinki. The protocol was approved by the ethics committee of the University of New Caledonia and the consultative ethics committee of New Caledonia (CEC-NC03-2016).Our study design is illustrated in Moreover, 3 weeks before the program, anthropometric and physical fitness data including aerobic capacity, speed, and agility were assessed during scheduled physical education classes as described below. Once anthropometry and physical fitness testing were completed, participants were equipped with research-grade activity sensors (GENEActiv) for 5 consecutive school days before the iEngage program to measure baseline PA behavior. They then started the iEngage program, which lasted for 4 weeks, with 2 1-hour modules per week. Throughout the program, they continuously wore a commercial activity tracker , which served as the educational support tool. The 2 activity trackers served very different purposes and collected PA data in different formats. To measure baseline PA, we needed a device meeting the research-grade criterion, and to support learning during the program, we needed a device that was commercially and easily available for future wide-scale deployment.The iEngage program targets health literacy, PA-related skills, and sugar-focused nutrition guidelines. Health literacy topics include the definition of PA, classifications of PA intensities , physical fitness parameters and their interpretation, rating one’s own effort during exercise, sedentary behaviors and their effects on health, health definition, well-being , sugar as a source of energy, and guidelines related to sugar consumption. iEngage v1.0 was delivered via an app (powered by BePatient) comprising 8 1-hour modules; it was self-paced and had learning activities as well as goal setting and self-assessment tasks. Each module proposed learning activities, quizzes, and brief 2- to 5-min PA sessions that generally focused on a particular series of movements: sprints, jumping, walking, running, squats, sit-and-reach, or push-ups. The total PA per adolescent over the 8 modules was 21 min; thus, these prescribed PA sessions had no significant direct impact on the MVPA levels and could not have contributed to an artificial increase in MVPA. PA was captured by the Misfit Shine 2 activity tracker and conveyed to the child via the Misfit app exclusively during the iEngage modules to support learning and provide help in self-assessing achievement against personal goals. At the end of each module, the adolescents were guided to set their own goals for the next module. These were 2-fold: first, they chose an individual “objective” in terms of number of steps and intended intensity , and then, they selected a “mission,” which was a specific task usually involving family, friends, or health professionals . At the start of the following module, they indicated whether they had achieved their goals. In the last module, they were encouraged to set long-term goals.A game challenge was designed to foster engagement in learning activities and goal achievement. The adolescents were assigned to 5 groups of 3 or 4, represented by an animal mascot. Groups won points when members achieved individual goals (objectives and missions) and for collective learning achievements during the modules; the results were tallied at the end of each week.Height was measured to the nearest 0.5 cm using a portable stadiometer . Body weight was determined using a scale  to the nearest 0.1 kg, with the adolescents in light clothing. Body mass index (BMI) was calculated by dividing mass in kilograms by height squared in meters. The BMI z score was then calculated. The percentage of fat body mass (FBM) was estimated from the skinfold thickness of the sum of 4 skin areas  measured on the right side of the body with Harpenden skinfold calipers, expressed in millimeters. This enabled us to determine the FBM and lean body mass) in kg. The detailed anthropometric methods are described in the study by Galy et al .t test was used to determine the adolescents’ agility. Trials were recorded using photocell gates in the same conditions as for the sprints. The maximal aerobic speed (MAS) test individually assessed running speed over gradually accelerating 1-min increments. When a child stopped the test at maximal effort, the last stage reached was recorded and converted to MAS as described in the study by Galy et al [Physical fitness tests included a time trial of 30-m sprints, with 5 m, 10 m, and 15 m lap times recorded using photocell gates  placed 1 m above the ground. The ly et al .Baseline PA was obtained via GENEActiv activity trackers . Means and SDs for the physical fitness tests were calculated for the whole group; these tests were MAS; speed at 5, 10, 15 m; and an agility test.The mean and SD of daily times spent being sedentary and engaged in light, moderate, and vigorous PA were calculated on the minimum bout-filtered daily accumulated data. Analyses were conducted using R Core Team 3.1.0. .daily behavioral clustering. We then devised a consistency rate function to track the regularity of the participants’ healthy PA behaviors over a sliding window of 7 days. We assumed that although the adolescents might not meet the PA recommendations every day, a noteworthy sign of progress is the fact that they meet them more regularly at the end of the program than at the beginning. These methods are summarized below.We analyzed the PA data in several ways to gain insight into the PA behavior changes during the program. We first identified the types of daily behavior observed through In addition to the binary achievement of minimum daily steps, we explored different shapes of the PA behaviors regarding the number of steps, the time spent at various intensities, and how these behaviors changed daily. The behavior clustering method thus revealed the types of PA behaviors over a day. The activity tracker data, for each child and each day, were mapped into vectors with 5 features: 1, 2, and 3—the total daily time spent in light, moderate, and vigorous PA, respectively; 4—the total number of steps; and 5—a binary feature that was true if at least 11,000 steps were accumulated on that day. Each of these vectors was, therefore, a representation of the PA behavior of 1 child on a given day. Numerical features were normalized.These vectors of daily PA behaviors were then clustered .n is the number of days , R(i) is the degree of target achievement on day i, and P(i) is a binary function showing whether behavioral data were available on day i.The consistency rate captures the frequency and regularity of desirable behaviors in relation to a specific target. It represents the average value of achievement of that target over a certain number of days, here 7, and is computed as shown in P(i) was to ensure that the data were only averaged for the number of days on which data were captured and to exclude those days when the adolescents did not wear their device . For instance, an adolescent who had not once met the recommendations over the prior 7 continuous days would have a rate of 0%, whereas one who did so consistently would have a rate of 100%. If no data were recorded in the 7-day period, the value was excluded. As this 7-day window moved each day, we were able to follow trends with the consistency rate.The purpose of To follow progress, we applied this method to 2 targets: (1) daily number of steps over 11,000 and (2) daily PA behavior within the desirable clusters. For the first target, R(i)=1 if the daily number of steps were achieved on day i, 0 otherwise. For the second one, each cluster was assigned a value within the range  corresponding to the activity level of that cluster , and R(i) was the value of the corresponding behavior cluster on day i.Descriptive data collected before the iEngage program are presented in Behavior clustering identified 3 types of daily PA behaviors, shown in Progress was assessed according to 2 daily targets: (1) reaching 11,000 steps and (2) being in the most active daily PA behavior cluster.Overall analysis showed that, on average, adolescents achieved 11,197 (SD 1376) steps per day during the 4-week program. The consistency rate of achieving 11,000 steps per day improved throughout the program . ConsistThe daily behavior clusters provided a more refined analysis of progress, taking into account the time spent at each PA intensity , the number of steps, and the achievement of daily recommended steps. With 3 clusters, R(n) was a 3-value function: 0 for cluster 0 (less active), 0.5 for cluster 1, and 1 for cluster 2 . A consistency value of 0 meant that every day of the past week was spent in the less active cluster, and a value of 1 meant that every day was spent in the very active cluster. A value of 0.5 could mean that the adolescents split their time between cluster 0 and cluster 2  or every day in cluster 1 (active but under the recommendations). The analysis of weekly consistency rate using these clusters is presented in The percentage of satisfaction with the modules  was 95% across the program . At the end of the program, adolescents were invited to set some goals for the future. Overall 85% (18/21) of adolescents declared intentions to achieve daily MVPA: 52% (11/21): “I intend to achieve 60 min of MVPA per day” and 33% (7/21) “I intend to achieve 30 min of MVPA per day”. Furthermore, 62% (13/21) declared “I will discuss with my family and friends about what I learned,” 62% (13/21) “I will seek assistance from close friends to reach my objectives,” and 0% “I don’t know/I am not motived.” To the question “Do you think you will achieve it?,” the answer was “Yes” for 38% (8/21) of the adolescents, “I’m not sure” for 57% (12/21), and “I don’t think so” for 4% (1/21).This study showed that deploying a technology-supported educational program in a school context targeting PA behaviors in a remote area of the PICTs is feasible. In addition, we provide new data mining techniques for tracking adolescents’ adherence to international PA recommendations through daily behavior clustering and consistency rate.The 95% Melanesian population was from a remote and rural area of New Caledonia. The overweight prevalence and the physical fitness level before the program  were conPA was analyzed using novel integrative approaches. First, it was analyzed per day and per adolescent rather than averaged over the observed period. This aligns with how recommendations are expressed and allowed the detection of changes on a daily basis. Second, a behavior clustering technique was used to combine the PA components to characterize the daily PA of an adolescent: volume (daily steps), time spent at each PA intensity, and achievement of internationally recommended daily steps. These objectively measured data enabled us to identify 3 daily PA behavior clusters : days spOverall, PA steadily increased, showing levels approaching the international recommendations, especially for those who were less active at the beginning of the study. We highlight that PA was realized outside the iEngage modules, ie during the adolescents’ daily life, as the program only prescribed at most 4 min per module (the remaining time in each session was spent learning via the app or discussing concepts with peers).Drehu) via videos that were incorporated into the digital program. This was particularly important because of the key role of spiritual leaders in Pacific communities [Drehu about the importance of PA and nutrition as ways to build healthy lifestyles and be part of healthy communities. We believe that this community involvement contributed to the adolescents’ strong appreciation for the program and their intentions to continue their engagement after the program end, and these too were positive outcomes of this exploratory study.The school educational team, including the principal, the teachers, and the school nurse, as well as the families and community (Melanesian custom) were very involved in the program, as we intended it. Indeed, a New Zealander study showed that contextualized learning facilitated knowledge translation in programs promoting scientific and health literacy . The adomunities . In partSome study limitations need to be underlined. This exploratory study was conducted in 1 class with 24 children for a period of 4 weeks and no control group was included, which limits any generalization. Although Misfit data were recorded continuously during the program, enabling us to obtain unique information on the children’s behavior, we did not measure the pre-post effects of the intervention via accelerometry, and this may have limited the interpretation. However, at the start of the program and for 5 consecutive days, the Misfit data were compared for data accuracy with the GENEActiv data. The daily totals were consistent, indicating that although the Misfit trackers sometimes underestimated small activity bouts, they still provided a close lower bound of the children’s activity.In conclusion, an integrative approach in the sports sciences, combining user-centered and community-based education and digital technologies (wearable trackers) to build health-related skills and change PA behaviors, seems promising in the PICTs. Our novel data mining technique provides an innovative way to assess adherence to international PA recommendations. Although several improvements are needed, this exploratory study showed that an electronic health program can be deployed in a remote Pacific area with a strong traditional culture and suggests the potential for implementing it at a larger scale for different ages, populations, and cultures.