Semillas de Apego , a community-based psychosocial support model for caregivers, materialized as a strategy to promote ECD in communities affected by violence and forced displacement.High quality investments during early childhood allow children to achieve their full potential by setting developmental foundations. However, challenges in the scale-up of evidence-based interventions make across-the-board implementation a non-trivial matter. Moreover, extreme contextual conditions -such as community violence, forced displacement, and poverty- impose a double threat. First, by directly affecting early childhood development (ECD), forced displacement and exposure to violence during early childhood, coupled with deficits in nurturing relationships, can trigger toxic stress, affecting children’s mental health and social and emotional learning. Second, contexts of extreme adversity exacerbate common implementation pitfalls in the scale-up of interventions. Recognizing and documenting “what it takes” to successfully implement “what works” can contribute to the expansion and effectiveness of evidence-based programs that promote ECD in these settings. This article presents the results of the process evaluation of SA during the 2018–2019 implementation in Tumaco, a violence ridden municipality in the south-west border of Colombia, South America. In this phase, the program reached 714 families, 82% direct victims of violence and 57% were internally displaced. The process evaluation combined qualitative and quantitative methodological approaches to produce evidence of factors that promoted implementation quality.Findings identified salient components of the program that promoted the program’s acceptability, adoption, appropriateness, fidelity and sustainability: a rigorous cultural adaptation; well-structured team selection and training methodologies; and a team support and supervision protocol to provide continuous capacity building and prevent burn-out and other occupational hazards common among professionals in mental health and psychosocial support interventions. The statistical analysis using monitoring data identified key predictors of the dosage delivered (a measure of fidelity). Evidence suggests that initial attendance to the program and observable characteristics -such as educational attainment, violence victimization and employment status-predict a successful compliance (in terms of dosage to benefit from the program).This study provides evidence for the development of structural, organizational, and procedural processes for the adoption, appropriate adaptation, and high-fidelity delivery of psychosocial support models delivered in territories affected by extreme adversity. Public policy efforts focused on early childhood are one of the most cost-effective mechanisms to end transmission of poverty across generations and successfully reduce socioeconomic inequalities globally . High quIt is estimated that, by 2021, a total of 36.6 million children in the world have been afflicted by forceful displacement caused by conflict, violence and other crises , and 449Semillas de Apego , a community-based psychosocial support model for caregivers, materialized as an intervention to promote ECD in communities affected by violence and forced displacement in Colombia. SA promotes maternal mental health through the improvement of healthy and nurturing child–parent relationships, which ultimately foster ECD and overall well-being. This article presents the results of the process evaluation of SA during the 2018–2019 implementation in Tumaco, a violence ridden municipality in the south-west border of Colombia. This work adds to the scarce literature identifying implementation barriers and enablers of ECD interventions in low-resourced settings such as Brazil  what factors should be considered essential during future scale-up phases to guarantee implementation quality?; and (3) are there any improvement opportunities in the implementation protocol to further promote implementation quality? Qualitative approaches were used to better understand specific dimensions of implementation quality – acceptability, adoption, appropriateness, feasibility, fidelity, penetration and sustainability-. Quantitative methods were used to describe mechanisms behind the level of take-up, adherence and “dosage” received by participants, as measures of fidelity in This study is conceptually framed by the final stage of the “translational pipeline” model, which collects research that focuses on understanding how to guarantee that an intervention with previously proven effectiveness will work at a larger scale, at similar settings  or at somehow different settings  . This stBy providing evidence on factors that promoted five of the abovementioned dimensions of implementation quality, this study informs the construction of structural, organizational, and procedural processes for future adaptation and high-fidelity delivery of psychosocial support models in territories affected by community violence and forced displacement. Promoting ECD and overall wellbeing of children living in similar contexts of extreme adversity should be one of the global key priorities, where the number of forcibly displaced families has more than doubled in the last decades and now includes over 30 million.Semillas de Apego (SA) promotes mental health and healthy child-caregiver attachment as a pathway for a proper development among children exposed to community violence and forced displacement. By providing psychosocial support to primary caregivers’, the program aims to help children reach their full potential amid toxic stress. SA’s theory of change is characterized by three premises and seven objectives that lead to three short-term outcomes and one long-term outcome see . The thrGiven these premises, the program’s curriculum is structured to achieve the following objectives: (1) generate capacity for self-reflection and a non-judgmental curiosity about own and children’s internal emotional universe; (2) promote sensory integration and adopt self-regulation and stress management tools; (3) raise awareness on the capability of resilience that caregivers and their children have; (4) give new meaning to past traumatic experiences and restore trust; (5) give a new meaning to childrearing support networks and build nurturing teams; (6) strengthen the relationship between caregivers and children; and (7) Increase the repertoire of context-relevant and culturally appropriate parenting strategies. As proposed by the theory of change, if these objectives are achieved, in the short-term SA should: (I) improve caregiver’s mental health, (II) build trust and strengthen childrearing support network and nurturing team, and (III) promote a strong and healthy emotional bond between the child and their caregivers. In turn, these transformations are expected to support the socioemotional and cognitive development of the child in the long run.Semillas de Apego builds upon the framework and structure of the Child–Parent Psychotherapy (CPP) (py (CPP)  and Builpy (CPP) . The CPPpy (CPP) , child mpy (CPP) , 2015, apy (CPP) . BuildinSemillas de Apego consists of 15 multi-caregiver group sessions, delivered once per week. Each session lasts approximately two and a half hours. Each group includes between 12 and 16 participants, all primary caregivers of at least one child 0–5 years old. Maintaining the same structure in all sessions and the same group composition makes the weekly meetings a predictable space for the participants, which generates a sense of safety and promotes trust and closeness among the group members. All the sessions have the following sequence of activities: a welcoming moment , a warm-up activity , a core activity  and a closing moment (to share experiences and collect learnings from the session). The main objective of the core activities for each session is broadly described in task shifting approach, which is a human resource management strategy originally developed to address public health crises in contexts where highly qualified professionals are scarce , the reflective supervision protocol is a fundamental mechanism to monitor the quality of the sessions. Second, this protocol creates safe spaces to address personal matters that can potentially affect the facilitators and could thus have a negative impact on implementation quality. In addition to providing continuing technical support, the supervision protocol aims to mitigate occupational hazards that are common among professionals working in the mental health field  (yndrome) .The final role in the implementation team is the “general technical director,” who ensures there is a successful adaptation and implementation of SA to the context where it will be delivered, oversees all technical issues (throughout implementation processes), leads the reflective supervision for the team of supervisors, and intermittently accompanies reflective supervision sessions for the facilitators. Additionally, the director oversees strengthening and managing the network of local institutional allies that support the implementation of SA, and reports to the program’s management team about the progress and challenges in the program implementation.During the 2018–2019 implementation phase, the program had the following support materials: the curriculum, a written document that presented the detailed the objectives each session, scripted all the activities in the session, described the needed arts, crafts and other materials; a training manual, which included the detailed workplan for the training of the facilitators and a summary of the theory supporting the design of the intervention; a technical guide, a written document summarizing theoretical and technical concepts related to the science supporting the program; and a simple guide for basic breath technics and mindfulness practices.Tumaco is a 200,000-inhabitant municipality that lies in the Pacific coast of Colombia, in the border with Ecuador. Historically, it has been a setting extremely affected by violence and poverty. In 2017, 243 homicides were reported in Tumaco, representing a rate per 100,000 inhabitants of 116.6. These violence figures exceed those recorded in the most violent cities in the world -such as Los Cabos (111.3), Caracas (111.2), and Acapulco (106.6) -, and isIn 2015, the team led a pilot study of SA in Bogotá with 64 participants (divided into three groups), all of them victims of forced displacement. The pilot, funded by the Ministry of Health and Social Protection (MoH) and the Interamerican Development Bank (IADB), was the first implementation of the SA curriculum and included a process evaluation and a results evaluation. This implementation phase was the result of a collaboration with the Mayor’s Office of Bogotá that aimed at integrating SA to the service portfolio offered to internally displaced families in the city. The pilot provided evidence that suggested the validity and appropriateness of the SA curriculum see .To further strengthen the program in the face of a future scale-up phase by identifying the program’s impact and implementation enablers and barriers, between 2018 and 2019 a process evaluation and an impact evaluation  were carPhenomenological Interviewing framework, aiming at understanding the factors associated with implementation success , through the testimonies of actors who had first-hand experiences in the program (N = 2). The second group of key informants were the team of program facilitators (N = 6), which was made up by community agents who were affiliated to Genesis Foundation, the NGO implementing the program. Interviews with members of these two groups aimed at collecting information on general implementation challenges, and focused on the appropriateness, feasibility, and potential penetration of SA. Finally, the third group included a sample of primary caregivers  that participated in the program. The interviews were designed to collect information about the implementation quality of the program. The interviews also explored the appropriateness, acceptability and adoption of the psychosocial and caregiving strategies discussed throughout the sessions. Each one of the interview guides explored the constructs described in Qualitative data was collected through Key Informant Interviews (KIIs) to explore the proposed research questions (see section 1). KIIs were guided by the  program . Semi-stAll interviews were conducted between December 2020 and February 2021. The interviews were recorded, transcribed by research assistants, organized and systematized using Nvivo Software®. The process of data analysis was done in two phases. Initially, the interviews were grouped together based on the key actor represented in each interview and were analyzed accordingly. Each question asked during the interviews was classified based on the constructs from the interview guide . In the The quantitative research design focuses on leveraging evidence on the fidelity of the implementation, defined as the extent to which SA was executed in Tumaco as it was prescribed in the original protocol . As explThe quantitative information used in this study was collected through a monitoring system designed for SA, which is structured and delivered through the digital platform KoboToolbox®. The system is comprised of custom-made digital forms that collect information provided by the program facilitators, after the execution of each of the 15 sessions in the program’s curriculum. Each form includes 4 to 5 open- and close-ended items that inquire about: achievement of goals or milestones in each weekly session and logistics . All program facilitators were trained and initially received weekly support to use the digital platform.At this stage, the monitoring system was able only to consistently collect information on participant attendance. Recurrent structural changes made to the monitoring system during the 2018–2019 implementation phase made it impossible to create indicators of adherence and quality of program delivery, across time and cohorts. Therefore, the quantitative analysis in this process evaluation study focuses on the degree to which SA was implemented as it was prescribed in the original protocol  in terms of the number of sessions delivered to each participant.time of survival measure (“t”) was created based on the information on the last session attended by each participant before either dropping out or successfully graduating from the program. A participant who continued in SA until session 15 was assigned a value of 15 (t = 15), while a participant who did not return to the program after session 6, was assigned a value of 6 (t = 6). Afterwards, two different dropout indicators were created using this time of survival measure. The first indicator, denominated “observed dropout,” is solely based on being present until the end of the program: all individuals who did not attend the 15th session are considered to have dropped out of the program. The second dropout indicator, the “normative dropout” measure, is based on a “minimum dose” that a person should receive in order to expect SA to have a significant impact. Thus, all individuals who did not attend at least 12 sessions are considered to have dropped out of the program. In the design phase, this threshold was defined by the technical team and is formally considered an “expulsion rule” in the set of norms of SA. Moreover, before program initiation, all potential participants are forewarned that if they miss three or more sessions, they will be asked to leave the program. Also, to capitalize the trust-building activities that are at the core of the initial weeks of SA, participants are cautioned that they would also be asked to leave if they miss more than one of the first 3 sessions of the program.In addition to descriptive analyses of the proposed dosage measures, a survival analysis see  was condThe objective of the survival analysis is to estimate the hazard ratios (HR) across different “participant profiles” using a Cox Proportional Hazards Model . To creaThe results of the process evaluation study aimed to identify information around seven spheres, namely: acceptability, adoption, appropriateness, feasibility, fidelity, penetration, and sustainability . Howeverreflective supervision protocol. The qualitative information collected provides evidence on the successful implementation of processes that play a key role to guarantee adherence and quality in the delivery (as explained in section 2): (i) recruitment, selection, and training of program facilitators; and (ii) the continuous support of facilitators.Evidence on the fidelity in this phase of implementation (2018–2019) of SA comes from both the quantitative and qualitative analyses. On the one hand, the quantitative approach provides evidence on the dosage  and the factors associated with a higher probability of compliance . On the other hand, results from the qualitative approach relate to the level of adherence to the protocol and the quality of the delivery. There were two mechanisms to monitor adherence and quality of delivery in the implementation phase this study evaluates. The first one was the monitoring system. The second mechanism was the The second measure, the take-up rate, is the proportion of the initially committed caregivers that attended to at least one session. For the total 2018–2019 implementation phase in Tumaco, 77% of the persons that enrolled, had an initial take-up of the program. An important result is that this proportion increased significantly, suggesting that important implementation barriers were surpassed from the second cohort onwards. This result is confirmed by the trajectory of the third dosage measure, the average number of sessions attended (after take-up): for the first cohort the average number of sessions was 7.6, while for the second, third and fourth it was 11.9, 11.0, 11.6, and 10.8, respectively. At an ideal level of fidelity (in terms of dosage) -that is, when SA is implemented as it was prescribed in the original protocol-, take-up rates would be 100% and the average number of sessions attended would be 15.observed dropouts” and “normative dropouts.” The distribution of these two indicators is used to capture the average level and variation of fidelity -i.e., how close was the implementation to the original protocol’s prescription in terms of the amount of program delivered or dosage-, overall and across participant profiles , and 54% did not attend at least 12 of the 15 sessions (“normative dropouts”).In the interviews with the technical team, they indicated that the recruitment and selection of facilitators was one of the main steps to ensure an effective training and the successful implementation of SA. The technical team agreed on three key factors to achieve a solid team: (i) doing face-to-face interviews with applicants, (ii) prioritizing the willingness of the candidates to deal with emotional and therapeutic processes and (iii) selecting persons that demonstrate honesty and transparency. One of the members of the technical team mentioned:“We decided to have a selection process. We held a group session where we provided them with a first experience of what the selection process would be like. This taught us a great deal! We received resumes that, given their academic training, made them seem as though they were the right fit. But we ended up choosing more intuitively and not according to the resumes, in favor of people whom we truly felt would be able to deal with the process”. BN Technical TeamAccording to the facilitators interviewed, the training for the program went beyond the initial training sessions and it implied constant learning. In fact, the training sessions and program reviews were recurrent; they took place once a week and included on-site visits by the team of supervisors, every 15 days. The team’s perception of the training is one of constant support and availability from the supervision team. Regarding training, the facilitators mentioned:“I feel that throughout all these years we have been constantly training because we never stop learning, there’s always something new. The people who have been in charge have always been very much focused on us as facilitators making use of all the tools so that we leave nothing behind.” Facilitator O“It’s wonderful, that we are not alone. We are always accompanied by them, by our supervisor, our coordinator. If something happens all of a sudden, they are around 24-7 and will always give us the support we need. I think we haven’t been left there, abandoned, they always give us their support”. Facilitator VThe facilitators also mentioned that one of the most effective learning experiences was participating in the program itself, as women and as mothers, before facilitating the sessions. In other words, the facilitators agreed that it was important to understand the program based on their own processes of childrearing and life experiences . This led the facilitators and participants in the program to establish relationships of empathy and trust, and to use the experience of the facilitators as real-life examples that can be achieved thanks to the processes proposed in SA. The facilitators referred only to the experiential processes as the most appropriate for achieving good implementation.“For example, in the activities that we carry out during the training, we include abuse, which is what I personally worked on with my son. I can now let go, speak, and recognize what I did with my son, which helps me to help mothers who will be entering the process, so that I can say: ‘Look, you can overcome this, you can transform this maltreatment into something positive for your child, in strengthening these relationships.’ Why? Because I went through this myself, I did it”. Facilitator“So, during the training, this could allow me to acknowledge my mistakes and where I am failing as a mother; this is a very specific tool; that helps mothers be open to this relationship, and they begin to believe in the facilitator. That is one of the marvelous touches of Semillas, the fact that they can see the facilitator as an average person who this also happens to and who has also gone through what they are experiencing. We do not judge them, nor do we look at them as if to say: ‘you are the worst mother’.” FacilitatorOur script is rigorous, but there are parts where you must provide an example from your personal life. We tell them: ‘I also beat them, I also did such and such, I was also violent, I did this...’. So, the mother feels that the person who is speaking is human, that we are connected.” Facilitator V“As mentioned before, SA adopted the reflective supervision model which provides a structure for the continuous training process of the facilitating team. Through individual and group supervision activities, facilitators received feedback from the technical team regarding the fidelity of the implementation of the curriculum, in terms of the adherence to the protocol and the quality of the delivery. According to the technical team, this reflective supervision takes place regularly and it considers the relationships between supervisors and professionals, between professionals and caregivers, and finally between caregivers and their children. A member of the technical team summarized the continuous support in the following way:In their first trip, the primary team began with the pilot project in Bogotá. It then continued its support for the team in this context, which we call reflexive supervision, in which they deal with clinical dilemmas regarding how to apply the model for different circumstances or groups and they are worked on collaboratively.” VR Technical Team““We think that this type of reflection and supervision, which is reflective, creates emotional support spaces that become part of a parallel process. And I think that they are critical in this type of work, because it evokes a lot in a person. One sees a lot of traumas, poverty, injustice. [...] We think that you need someone a bit more distanced from the system who can help you think about what you can come and say, ‘Oh, I feel this way!’. And you can receive this type of support”. VR Technical Teamreflective supervision is conceived to provide technical support, but also as a strategy of psycho-emotional well-being for professionals who work with families and children in highly adverse contexts. According to the interviews, this type of supervision enables the team to manage strong emotions and carry out reflexive actions to tailor the implementation practices. The facilitators discussed at great length the benefits of reflective supervision by stating:As stated by the technical team, “Reflective group supervision is about how to understand or ask how the group is doing. And when a particular circumstance, situation or weakness is generated, it must be strengthened through exercises, breathing, workshops, or homework that they assign to us. So, it’s wonderful to see how we are growing little by little and strengthen things that we were perhaps not so strong in, even theoretical things.” Facilitator M“Well, at the individual level, we talk about more personal and deeper matters that sometimes affect both the work and the person. So, it is more a matter of getting deeper into personal matters, asking: ‘how do we feel? Did it help the session run smoothly? Did it prevent us from carrying out the session […]? What is happening in our lives at that moment?’ That kind of things.” Facilitator OThe interviews with the facilitators further revealed that reflective supervision is a model that favors professional well-being and allows the team to feel recognized from the emotional perspective, which helps them not only to deliver the sessions with the expected quality but also to improve their responses to conflict in their family and daily lives.“I have never had this kind of attention towards myself. Do you understand? It was also a matter of picking up the form, going out there, applying and taking it and getting things done. They are concerned about us here, how we are doing, our relationships with our colleagues, bosses, children, family; it’s comprehensive. To me this is a beautiful aspect of the program and hopefully it will never change, the end goal, of not just dealing with the community or the mothers, or the people that we’ll be working with, the children, early childhood, but also with the facilitator who can also recharge and be nourished by the experiences projected by the community.” Facilitator O“One of the training sessions was very intense. It focused on [past] trauma. The things I carried with me from childhood, and they made me see that the world can be different. And they made me start to change patterns that I had formed in my childhood, negative things, things that are not good for me. And the therapy that they performed on us during the training sessions helped me a great deal. I had fallen into a very similar pattern to my mother’s. It was not a positive thing, it was negative. And I really understood that we can change those child-rearing patterns and be different. I do not want to be like my mother, I want to be a different person. And that was one of the things for which I am most grateful to the supervisor and to the program, because I really cut out that pattern and now, I am another person, with different qualities. This left a mark on me and helps me when I am in the field, and I am working with my group. To speak to them based on my own example, the experiences I had and went through to transmit it to them.” Facilitator GAppropriateness refers to the relevance the intervention has among providers and/or participants . In thisThrough the interviews with facilitators, it became evident that the SA curriculum allows for flexibility to recognize and integrate the contextual and cultural factors of the community within which the program is being implemented. Although the program’s curriculum has core components that should not be modified, the implementation protocol of SA explicitly recognizes that certain components must be adapted to the local context. This adaptation processes involves collaborating with community agents and recognizing their knowledge and understanding the social, economic, cultural, and political realities of the context. It was also necessary to consider different pedagogical strategies in order to teach the content of the manual to the facilitators. In the case of Tumaco, the technical team mentioned that an “experiential learning” methodology was necessary to effectively train the local team of program facilitators. The team arrived at this approach after an initial -and mostly unsuccessful- attempt to do the training based on autonomous reading of the program’s manual and additional supporting lectures. Members of the technical team mentioned:“The team (in Tumaco), for example, had not read the curriculum. This was a challenge that we faced in the beginning because there was a cultural difference in the context of Tumaco. […] We decided in our consultation that instead of forcing our way of working, which mostly comes from Bogotá, why don’t we learn to work as they do? Maybe it’s more organic and maybe it’s not a matter of reading. Since they are not going to read what is assigned to them, what if change our methodology to match theirs? What if we go at their pace instead of imposing our own? We should not claim that they are not paying attention or that they don’t care, when in reality, something else is happening there.” VR Technical Team“The arrival of Semillas de Apego in Tumaco involved various trial and error exercises through which the technical team reached the conclusion that the team should be trained in an experiential manner. The importance of living the program, of incorporating the tools in the day-to-day lives of the facilitators, was consolidated as one of the criteria for training the local teams on the theoretical contents and group management and including it as a factor to consider in training for adults” BN Technical TeamFacilitators also agreed that acknowledging cultural differences was crucial for the delivery of SA in Tumaco. Most cultural changes had to do with the integration of colloquial language that people could easily understand. But also, the cultural adaptation implied a consideration on how the traditions of the Colombian Pacific could be integrated to the curriculum to relate its content and objectives to the culture of the participants. For instance, oral traditions of Afro-Colombians were integrated through children’s songs and lullabies that are very present in local child-rearing practices . One of “The first part was the most difficult. It involved dealing with this new type of work. So, it was about the approach and how to reach this community. Because sometimes it is not just a matter of looking at how the script [in the curriculum] is structured. No, you must speak as they do there, using colloquialisms, words that are spoken in my community and that we are familiar with. Because if you use very technical words in the mother’s group, they will not understand you. So, it is better to follow the script but to change that ‘particular word.’ The way we communicate among ourselves here in our territory and bring it down to that level so that they can understand. Because there are questions in some parts of the script where, if you ask the mothers the way it is written in the script, they will not understand. But we know how to change it, so that it will end up being the same question, but using words spoken here in this environment.” Facilitator GAlso, adaptations to the context included changes in the music and materials used during arts and crafts workshops. The implementation team incorporated traditional instruments and music from Tumaco, which generated engagement and familiarity in the participants. Moreover, in an effort to reduce costs and environmental impacts, the implementation team incorporated materials endemic to the region to substitute the ones that had to be imported from other places.“We have done many things to adapt the program to the context, above all regarding the music that is played during the sessions, which we have been improving. We have been including things that are originally from the Pacific in the topics of the sessions. Things such as materials, or things like that, that can be obtained here, that we have been including, and this has also been helpful”. Facilitator OInterviews with participants provided important evidence on the acceptability of SA, in terms of how satisfied the caregivers were in the program, and if they perceived that the curriculum addressed their needs. For instance, caregivers highlighted how the program helped them to increase their capacity to regulate their emotions, understand the developmental needs of their children and increase the ability to interact positively with them. Participants mentioned:"It was a very good program, one that helps moms -not only new moms- to try to control their anger. Because all of us can suddenly become desperate. One has to be honest; it happens to everybody. But to control your breathing and say ‘well, just breathe’”. Participant A.“It’s not that I'm the most explosive person. But [the program] taught you to control yourself, to calm down when the children are getting frantic, and to correct them with patience and to be a little more tolerant”. Participant I.Through our data collected from mothers, fathers, and caregivers, it became clear that although the incentives to attend the sessions were motivating - as participants could buy food and necessary items for their children - their main motivation were the benefits obtained from the program. We believe this is evidence that also supports the acceptability of the program. Program participants frequently mentioned this:“Yes, there was an incentive. They gave me 20,000 pesos [$4 USD] for each session. It [the money] was for the girl or whatever I wanted. […] But for me, the most important thing was not the money, but the learning received from this type of training, right? The most important thing for me was the training." Participant B“I went mostly to receive the talks, which taught us useful things. But the money was also useful because I went to the supermarket and bought something for the kid or whatever I needed. As a single mother, everything helps me.” Participant FThrough the interviews, participants also shared how they integrated into their lives tools and strategies provided by SA. The adoption of self-care practices and parenting practices strongly suggests not only that the program is relevant for the context, but also that it can be successfully implemented.In particular, frequently caregivers explained how their intention to change their way of interacting with their children is a consequence of them understanding the potential positive impact this has on the behavior, wellbeing, and development of their children.“It is also important to place ourselves in the child’s perspective and see how we can educate them in a way that does not affect their development. They are in that stage of development, and they are discovering everything. But we also must instill in them what is good and what is bad. Because why are we going to lie and say ‘this is good’, when it is bad […]. My girl, she likes the phone a lot, so I told my husband not to lend her the phone so much because every day she asks ‘mommy the phone’. She wants to be stuck watching videos of dolls all the time. So, I had to control those things more and try to teach them other types of games”. Participant E“Sitting down to talk and then use the techniques they gave us there [in the program]. We are adults, obviously we are tall... but sitting or squatting down to’he child's height so that the child looks at us in the same position […] at that moment, we can talk in front of’he child's eyes and tell the child what is wrong and why should correct it. Then she will understand […]”. Participant OParticipating in SA also helped to expand the caregivers’ comprehension regarding child development and to prioritize their needs as individuals. A participant mother mentioned:“There were many things that I ‘skipped’ or did my own way, because I thought that was the way to do it. In the program, they taught us how things should be in raising children. Because they are a reflection of bad parenting. [Before SA] my son, when he ate, would induce vomiting. And they explained to me that this happened if I forced him to eat and suddenly yelled at him, attacked him: 'eat, eat, eat' - it was something that the child did because I attacked him. All of that helped me with my kid. I was suffocating him too much at mealtime.” Participant CAdditionally, program participants highlighted, thanks to the program, they were willing to share and create new spaces to support peers, thus creating new networks. Essentially, the participants expressed that they were open to build new relationships in their communities as a consequence of their participation in SA.Well, uh, sharing! Sharing with others, talking to others. Being more of a neighbor to my neighbor, being more attentive to my neighbor, who may be living in a worse situation than I am, and putting into practice what Semillas de Apego taught me” Participant A"We shared a lot among all those who went to the meetings. But more than anything, we got to know each other." Participant G.According to the interviews with participants, the activities related to mindfulness were the most memorable aspect of the program. Participants mentioned that mindful breathing exercises were particularly effective for managing stress and promoting relaxation. The participants highly valued practicing mindful breathing and recognized that this skill helped them improve their parenting skills and the communication with their children.“Well, the truth is, one of the things is learning to breathe, right? When you feel, you are going to explode! So, you go - think about it before doing it. You think and breathe, and it calms you down!” Participant G“The breathing part. Learning to breathe. Not only to breathe when you have problems with the children but also when you have many problems in your head. So, learning to breathe well, to breathe deeply! Participant E“The way of breathing at the moment when the children drive you crazy. This therapy of how to breathe to calm down.” Participant OThe process evaluation found that the sustainability  of SA faces at least 3 important challenges: (1) guaranteeing the safety of the team, (2) procuring local support and commitment, and (3) ensuring funding to expand and maintain operations.Tumaco is a highly complex territory where there is an amalgamation of social issues, including the presence of illegal armed groups, illegal drug trafficking, and forced recruitment and displacement. Confrontations between the existing legal and illegal armed groups generate invisible barriers at the territorial level. Therefore, entry into the neighborhoods posed the question if it was really possible to guarantee the safety of the implementation team. A facilitator mentioned:“Another challenge, that is also important, was the fear that I suppose many of us feel when entering certain places where violence is present. Where you didn’t know whether it was better to go in or not to go in. But love for the program, as well as our professionalism, helped us to keep moving forward”. Facilitator OAdditionally, the interviewees mentioned that the necessity of developing strong partnerships and communication mechanisms with the communities where the program was going to be implemented to have their buy-in and support, prior to start any work. The facilitators highlighted the importance to engage key community stakeholders such as teachers, educators, government employees or other agents recognized and respected by the community. Facilitators underlined how constant interaction and communication with the community leaders helps to mitigate security risks.“We were going to the children’s kindergarten. ‘They’ [illegal armed-group members] knew that. We went in with our [program-labelled] vests; we brought our ID. They knew that we were there to help, contributing to the community, to the children. In other words, ‘They’ don’t interfere with aspects involving children. ‘They’ might get involved in other people’s affairs, but not the children’s: they also take care of the children. ‘They’ knew that we were not doing anything wrong, we weren’t spies of any kind or infiltrators; we were there to help the families, the children.” Facilitator HThe evidence gathered in the conversations with the team of facilitators revealed a few key challenges when scaling-out SA in other municipalities or territories. First, the facilitators mentioned establishing partnerships at the institutional level to ensure visibility and a better coordination of the program. A facilitator mentioned:“I think that the most important thing would be to foster connections with the main entity or institution in the municipality. Because we can’t arrive at a municipality with a program that no one knows about and simply go in and say: ‘okay, I’m going to work here and that’s that’. The idea is to foster those connections or partnerships once we arrive, similarly to when we arrive in the neighborhoods through the community action board. So, there we would connect with the Mayor and the [government] entities that exist in the territory. How can we discuss [with these partners about] the program? Obviously by […] asking for their perspectives about the municipality […]; what they think, whether they approve of the project taking place in the municipality.” Facilitator OAccording to the technical team, the future donors of SA must take on a long-term commitment, of at least 3 years. That is the approximate time to reach the territories, adapt the curriculum to the context, and achieve the minimum quality level of training and implementation.“Well, the funding has always been… It’s a challenge for any program, right? And that determines many things, because once you are allied with the person that provides funding, well, after that we become responsible for what they want and how they want to measure progress and all of that, so having the flexibility to have funding that includes the level of support that we know works and reflective supervision is essential. These resources are necessary but worthwhile! We know that it has been a challenge to find partners that believe in this work and who want to develop high quality. And that it would be, a kind of commitment. We know that it takes time and that it would be a three-year commitment. At least three years, because normally the first year involves great efforts to connect with key people to start the training. It took pretty much that much time for them to truly internalize this, experience it, make it their own.” VR Technical TeamSA aims to fill a salient void in the portfolio of ECD services in Colombia, Latin America and other regions with ongoing armed conflicts or persistently high levels of community violence. Given the devastating effects of early exposure to violence, there is an urgency to expand the reach of evidence-based programs that can effectively promote resilience among families rearing children in contexts extreme adversity . BuildinThe process evaluation of the 2018–2019 phase of SA in Tumaco contributes to future efforts to expand ECD programs by advancing in the understanding of “threats to scalability” and enablers of “implementation success.” The evidence provided by this study suggests that the program’s curriculum is perceived as relevant and fit to the reality of participants, that caregivers have the intention to adopt the provided self-care and parenting tools, and that the overall level of fidelity is acceptable. Moreover, this study identifies two key implementation enablers that will be extrapolated to future scale-up and scale-out phases of SA, and that can be valuable to other ECD programs that aim at thriving in similar contexts.The first implementation enabler was the intentional flexibility of the program for its adaptation to the context and culture. It is important to highlight that this adaptability is intentional, and follows a structured process that defines the contents in the curriculum that: (a) are core elements and “should never be adjusted” to the context; (b) “must be contextualized” every time the program lands into a new community, to promote appropriateness, adherence and effectiveness; and (c) “could be adapted,” if the implementation team perceives that the adjustment will improve the level of implementation success. This evaluation shows that a thorough and structured cultural adaptation is a strategic practice that promotes implementation success, and probably is also at the core of the program’s intervention success.task shifting approach. Previous studies have demonstrated that task-shifting is particularly important and valuable in settings with few professionals available  and, given the widespread of diverse expressions of violence, there is relatively large population of victimized families and children. Given that they already are part of the community, training lay health workers is particularly important because the intervention fosters an already existing trust and rapport with the participants, which is a highly important factor in conflict-affected areas where it is difficult to trust outsiders. Yet, it is important to highlight that the collected evidence suggests that the task-shifting model requires a robust process of recruitment, selection, and training of facilitators, and a structured mechanism that provides continuous support of program facilitators, such as the reflective supervision protocol.Despite the encouraging findings, the study found that SA still has at least four areas of improvement. First, the safety of the work teams remains a concern in the implementation and scaling-up of the intervention. Previous studies have indicated that in contexts of violence, safety has consistently come up as a central topic in the ethics committees and has been subject to much supervision by universities and organizations . While TSecond, in terms of fidelity there is an important “learning curve” between the first time the program is implemented in a territory and the subsequent iterations. Evidence from the 2018–2019 implementation shows the dosage delivered to participants  was much lower for the first cohort. This probably is the result of a combination of implementation threats, such as: lack of engagement and trust from the community and key allies , inadequate training of the team, insufficiently deep adaptations to the context and culture, among other factors.Third, the program needs to integrate differentiated strategies to prevent the low-dosage and higher drop-out rates of particularly vulnerable participants. For instance, the results from the statistical analysis show that participants that are less educated and have been direct victims of violence systematically have higher odds of dropping out of the program and not participating in enough sessions to benefit from the program. Improving the compliance of participants and attaining the minimum planned dosage for most  the participants that have an initial take-up is a direct way of improving the cost-effectiveness of the program.The fourth area of improvement is related to the strengthening of the program’s monitoring system. In the evaluated phase of implementation (2018–2019), SA did not have any direct feedback channels with caregivers  to monitor program engagement and perception of the outcome. This is a key issue that should be addressed as a future improvement. An additional, and probably more challenging improvement opportunity for the monitoring system would be to develop an analytical tool to systematize the information exchanged in the reflective supervision sessions.A recommended strategy to integrate the lessons learned in the first implementation efforts in Tumaco and prepare to expand the program to other communities, is to explicitly develop protocols around the four implementation stages proposed in the “EPIS conceptual model” : (E)xploparticipatory action approach by including caregivers in the study design.Future implementation science studies could contribute by advancing designing and testing strategies to minimize the “learning curve” when deploying an evidence-based intervention in a new context. Future studies could also provide evidence on how to increase the cost-effectiveness of the program, which is a non-trivial determinant of scale-up efforts. Finally, subsequent research could integrate more the voices of participants to better understand their experience with the intervention and even have a The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.All study procedures were reviewed and approved by the Institutional Review Board of Universidad de los Andes . The participants provided their written informed consent to participate in this study.AHR led the development of the field work and led the design and development of the manuscript. AHR conducted the analysis of the quantitative data. NCF and MP-L conducted the analysis of the qualitative data. AHR, NCF, and MP-L drafted the manuscript. All authors collectively conceived and conceptualized this original research article, contributed to the discussion, reviewed the manuscript, and provided critical feedback.We acknowledge generous financial support the Saving Brains partnership (Grant Agreement ID SB-POC-1809-19091), United Way Colombia, Fundación Exito, Fundacion FEMSA, Genesis Foundation, Fundación Coca-Cola, Primero lo Primero, and the Facultad de Typo: Economía at Universidad de los Andes.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


To assess the availability and uniformity of application information for away subinternships and survey 4th‐year medical students on their experiences obtaining away subinternships in otolaryngology‐head and neck surgery (OHNS) during the 2022 to 2023 application cycle.Cross‐sectional study.Online survey.The Association of American Medical College's Visiting Student Learning Opportunities (VSLO) program was queried for information on OHNS away subinternship applications. A survey assessing 4th‐year medical students' perceptions of the away subinternship application process was distributed via OHNS residency program directors and Otomatch.Of 129 OHNS residency programs, 103 (80%) offered away subinternship opportunities on VSLO. Variability in application release dates , offer release dates , and estimated cost ($22‐$5500) were found. The most common application requirements were a transcript (98.1%) and a CV/resume (90.3%). There were 64 survey respondents, for a 13% response rate. The most common concerns include applying to too few programs (80%) and not knowing offer release dates (77%). The most common stressors include choosing a number of programs to which to apply (48%) and cost (35%). The majority (76%) reported difficulty finding updated information on program websites. Among the proposed changes, the greatest support was found for having all applications on VSLO (88%), uniform application release date (84%), and uniform application requirements (82%).The OHNS away subinternship application process is a significant source of anxiety for medical students due to the tremendous variability in application and acceptance procedures. Having all applications on VSLO, uniform application requirements, and uniform application opening and offer release dates would better facilitate this process. Otolaryngology‐head and neck surgery (OHNS) is among the most competitive specialties. In the 2022 Match cycle, the National Resident Matching Program reported that 574 medical students applied for 361 residency positions, yielding a 63% match rate.Away subinternships are a staple of the 4th‐year medical school curriculum, especially for students applying to highly competitive specialties. These 2‐ to 4‐week rotations allow for educational experiences outside students' home institutions and the chance for both programs and applicants to assess “fit.” Programs consistently report personal knowledge of applicants as among the most important criteria used when ranking applicants, and in cases of exceptional “fit,” match rates as high as nearly 50% are reported.Recent studies have characterized away subinternships in several fields, including orthopedic surgery, emergency medicine, plastic surgery, and OHNS.Using the Accreditation Council for Graduate Medical Education (ACGME) list of 129 accredited US OHNS residency programs, we queried the AAMC's VSLO website on June 9, 2022 for away subinternships offered by each program. We performed the search on VSLO under the “Find Electives” tab with a filter added for Specialty: Otolaryngology. In the search bar, we searched for electives offered by each program either by the program's name as it is listed on the ACGME list or its location .For programs with away subinternships on VSLO, data were collected regarding application timelines, offer release dates, processing or tuition fees, total estimated cost , and points of contact. Each application was also queried for program‐specific requirements which included: CV/resume, transcript, USMLE scores, letter of recommendation, personal statement, AAMC immunization form, immunization titers, current tuberculosis (TB) test results, mask fit results, photograph, background check, basic life support/advanced cardiac life support certification, drug screen results, malpractice insurance, and personal health insurance.Electives offered by programs not utilizing VSLO were found using the Google search “[program name] away subinternship application.” Information about these electives was gathered, including whether the program has a specific webpage and application system.Survey questions assessing students' perceptions of the away subinternship application process were developed after a literature review and discussions with current otolaryngology applicants at the Pritzker School of Medicine, University of Chicago. Our survey assessed student demographic characteristics , application process characteristics , and areas of improvement. We piloted the survey with 5 senior medical students, and their feedback on clarity, appropriateness, and organization was incorporated. The final, anonymous electronic survey consisted of 46 items with quantitative and qualitative questions, including the Likert scale, multiple choice, and free‐response questions. The institutional review board at the University of Chicago (IRB22‐0736) approved the study .In May 2022, we distributed the survey to a national electronic listserv for otolaryngology residency program directors who were asked to email the survey link to their respective senior medical students. We also posted the survey on Otomatch, an online forum for otolaryngology applicants. Reminders were sent to both platforms after 2 weeks. The survey was open for 4 weeks between May and June 2022. All US senior medical students in allopathic or osteopathic medical schools applying to otolaryngology residency programs during the 2022 to 2023 application cycle were eligible. Participation was voluntary, and data was collected using REDCap, a secure application for managing online surveys .T tests and 1‐way analysis of variance examined associations between application process stressors with covariates such as race, gender, and access to home ENT program after converting ordinal data to numbers . Two authors (T.C. and S.T.) independently generated common themes from survey free‐response questions. All authors finalized themes, and theme frequencies were calculated using keywords related to a theme. All analyses were performed in STATA 16.0 , with a p < .05 determined as significant.Standard descriptive statistics were used for analysis. Only completed surveys from 4th‐year otolaryngology‐bound US medical students were considered. Of 129 accredited US OHNS residency programs, 103 (80%) offered away subinternships on VSLO. Of these electives, the dates that applications opened ranged from January 18, 2022 to June 3, 2022, with the most frequent date being April 1, 2022 . Offer release dates ranged from January 27, 2022 to August 7, 2022, with the most frequent date being May 2, 2022  , CV/resume , photograph , AAMC immunization form , and USMLE scores . The least frequently required application materials were current TB test , letter of recommendation , drug screen , immunization titers , and mask fit   OHNS residency programs were not found to offer away subinternships on VSLO. Using the search criteria outlined above, 23 (88.5%) of these programs had a webpage for away subinternship opportunities, and 14 (53.8%) had a specific application. Of the programs without electives on VSLO, 13 (50%) are affiliated with either military or osteopathic schools. Notably, several well‐known OHNS residency programs were found to utilize their own application systems for away subinternships, including Johns Hopkins University, Stanford University, University of Iowa, University of Pennsylvania, Cleveland Clinic Foundation, and NYU Grossman School of Medicine.Electronic Residency Application Service (ERAS) preliminary data suggested an estimated total of 490 US MD or DO OHNS applicants for the 2023 Match cycle. A total of 64 surveys were completed and included in our analysis, yielding an estimated 13% response rate. While age and gender were representatives of national medical school enrollment data, our study had more Asian students (44%), fewer black students (3.1%), and fewer students from the AAMC Western region (14.1%).Table Applying to too few programs (80%) was the most commonly selected concern while applying for away subinternships. Other concerns included not knowing offer release dates (77%), not receiving enough offers (73%), applying to too many programs (61%), and overall cost (44%). Notably, not knowing offer release dates (22%) was the most frequently ranked number 1 concern, followed by applying to too few programs (19%) and applying to too many programs (16%).Figure p = .88), race /mixed, p = .81), and access to home ENT program  numerically converted stress levels due to preparing application materials was found by gender , race , and access to home ENT program . Applicants reported relatively less stress with regards to choosing (25%) and ranking (22%) available rotation dates.Figure Frequencies of various application process events are displayed in Figure Figure Free‐response comments from 21 applicants demonstrated similar themes regarding areas of frustration with the away subinternship application process. Themes included not knowing offer release dates (52%), fear of being “blacklisted” from programs after withdrawing applications and/or rejecting offers (29%), concerns about rotation logistics such as cost and housing (24%), outdated program websites (15%), variability in application requirements (14%), and variability in application release dates (14%).OHNS is a competitive specialty. Away subinternships allow programs and applicants to assess “fit” and increase the likelihood of a successful match. In a recent study, 100% of surveyed 4th‐year medical students interested in OHNS from 10 US medical schools completed an away subinternship.The AAMC's VSLO program was created, in part, to simplify the away subinternship application process. However, despite 80% of OHNS programs utilizing VSLO, many survey respondents commented on the difficulty of finding accurate information regarding elective opportunities, including offer release dates and program websites. A portion of students' dissatisfaction with the application process may be attributed to the 20% of OHNS programs that have yet to utilize VSLO, around half of which are military or osteopathic residency programs. While military programs utilize their own vetting process, which VSLO may not be suited for, it is unclear why osteopathic programs underutilize VSLO. Additionally, several allopathic residency programs do not utilize VSLO, including programs that traditionally receive abundant student interest, highlighting the need for streamlining access. Unsurprisingly, when asked about potential changes, having all applications for away subinternships available on VSLO was supported by most respondents (88%).Among programs that utilize VSLO, varying application requirements, deadlines, and fees contribute to increased applicant stress. Respondents indicated most difficulty finding when applications open and when offers are released. Our analysis found widely varying application release dates and requirements, suggesting that application contents still differ tremendously among programs utilizing VSLO. Moreover, many applications on VSLO contain outdated information and schedules, including rotation calendars and rotation sites, requiring prospective applicants to search for more accurate information on program‐specific websites that are often also outdated. Unfortunately, inadequate program webpages are not unique to OHNS, suggesting that programs across all specialties must continually update webpages to ensure their accuracy and utility for prospective students.Applicants also expressed frustration with the lack of transparency. Even when application and offer release dates were published, a substantial number of respondents reported applications opening and offers being released on dates different than initially indicated. Especially concerning is that 11% of applicants reported almost never hearing back from programs regarding their application status. Unsurprisingly, the most commonly reported concerns and stressors are regarding the number of applications to submit to guarantee enough offers in the setting of not knowing when offers would be released. Given the importance of away subinternships in the Match process and the competitiveness of the specialty, students often apply to many programs to optimize their chances of securing away subinternships. In our survey, applicants applied to up to 12 programs, despite wanting to do a maximum of 4 away subinternships. Because application submission status is often not known, students focus on securing any away subinternship rather than one(s) that are potentially a good “fit” . If a less desired program notifies applicants first, applicants must either accept, foregoing a better “fit,” or decline, possibly alienating that program in the Match with no guarantee of acceptance from other, more desired away subinternships.The lack of transparency, uniformity, and availability of accurate away subinternship information makes planning difficult for prospective applicants. A significant number of surveyed respondents favored uniform dates for when applications open and for when offers are released in addition to uniform application requirements. A recent study highlighted the successful implementation of standardized deadlines within the OB‐GYN residency application process, which was well‐received by applicants and programs alike.Although it may be possible to standardize both when applications open and when offers are released, creating uniform application requirements poses a larger challenge given hospital‐specific requirements, such as drug screens and TB testing. While hospital and state policies for healthcare workers may be difficult to change, requirements like letters of recommendation and personal statements are program‐specific and lend themselves more nicely to standardization. Such requirements add significant time and effort to an already stressful process that occurs during a busy time in medical school. Many surveyed respondents suggested that these requirements may dissuade them from applying to particular electives. In as much as standardizing application requirements is feasible per specific medical school requirements, it should be a serious consideration toward facilitating the process for applicants.Our study has several limitations. Despite efforts to broadly distribute our survey through Otomatch and program directors, our response rate is low at 13%. Furthermore, the true response rate is difficult to measure as complete applicant data for the 2023 Match cycle were not yet available, and we utilized ERAS 2023 preliminary data as an estimate. If the number of applicants during the current cycle was lower than the previous one, the percent response rate could actually be higher. Our survey was administered in May 2022, potentially limiting applicants' recall of the application process, which typically occurs in March and April. Additionally, we could not differentiate DO and MD respondents, precluding assessment of factors specifically affecting DO applicants. Another potential limitation related to the way the survey was disseminated is that the survey under‐represents applicants with no otolaryngology home program. As with all survey‐based studies, sample bias and response bias potentially limit the generalizability of our results. Programs may have had more information available regarding away subinternship application processes and requirements, including applications available on VSLO or separate websites, but we could not find them. Beyond VSLO and residency program websites, applicants could have gathered information about the application process from other sources, including mentorship from faculty and senior students and online forums, like Otomatch, all of which our study did not assess.The otolaryngology away subinternship application process is a considerable source of anxiety for medical student applicants, largely due to the tremendous variability in application and acceptance procedures. We recommend the following changes to improve the away subinternship application process: all applications on VSLO, uniform application requirements, and uniform application opening and offer release dates. While these changes require programs to adapt their current procedures, recent changes to residency application processes, including uniform interview offer dates and signaling, suggest that change is possible. The proposed recommendations would increase uniformity and transparency and could allow applicants to make more thoughtful decisions based on their priorities, ultimately benefiting both applicants and otolaryngology residency programs.Thomas Cyberski, design, conduct, analysis, and presentation; Swetha Tatineni, design, conduct, analysis, and presentation; Andrea Shogan, design, analysis, and presentation; Fuad M. Baroody, design, analysis, and presentation.None.None.Survey questions assessing students' perceptions of the away sub‐internship application process.Click here for additional data file.


Increases in solute transport were consistent with the effects of enzymatic treatment on CEP composition, which included reductions in sGAG content of 33.5% (human) and 40% (bovine). Whole disc biomechanical behavior—namely, creep strain and disc modulus—was similar between discs with enzyme- and buffer-treated CEPs. Taken together, these findings demonstrate the potential for matrix modification of the CEP to improve the transport of small solutes into whole intact discs.Poor nutrient transport through the cartilage endplate (CEP) is a key factor in the etiology of intervertebral disc degeneration and may hinder the efficacy of biologic strategies for disc regeneration. Yet, there are currently no treatments for improving nutrient transport through the CEP. In this study we tested whether intradiscal delivery of a matrix-modifying enzyme to the CEP improves solute transport into whole human and bovine discs. Ten human lumbar motion segments harvested from five fresh cadaveric spines (38–66 years old) and nine bovine coccygeal motion segments harvested from three adult steers were treated intradiscally either with collagenase enzyme or control buffer that was loaded in alginate carrier. Motion segments were then incubated for 18 h at 37 °C, the bony endplates removed, and the isolated discs were compressed under static (0.2 MPa) and cyclic  loads while submerged in fluorescein tracer solution . Fluorescein concentrations from site-matched nucleus pulposus (NP) samples were compared between discs. CEP samples from each disc were digested and assayed for sulfated glycosaminoglycan (sGAG) and collagen contents. Results showed that enzymatic treatment of the CEP dramatically enhanced small solute transport into the disc. Discs with enzyme-treated CEPs had up to 10.8-fold (human) and 14.0-fold (bovine) higher fluorescein concentration in the NP compared to site-matched locations in discs with buffer-treated CEPs ( Back pain is the leading cause of disability and is a major public health problem worldwide . In certProper disc nutrition involves nutrient and metabolite exchange between the nucleus pulposus (NP) cells and vertebral capillaries , and sevMotivated by those findings, we explored a matrix modification strategy for improving disc nutrition that involves enzymatically reducing the amounts of collagen and aggrecan in the impermeable and fibrotic CEPs . TreatmeNine intact bovine coccygeal motion segments were harvested from three oxtails (3 segments per oxtail) acquired within 2 h of slaughter from a local abattoir (approximate age range of the steers: 18–22 months old). After harvesting the motion segments, a 17G Crawford needle with a plasma-mediated coblation wand (ArthroCare 2000) was inserted through the posteriolateral annulus directed toward one CEP in each disc, the wand was energized on a low-power setting (2 W) for 10 s as the needle was advanced to the distal annulus, and the coblated tissue was debrided with PBS. This coblation step was necessary for overcoming the high back pressure in the otherwise healthy bovine discs. Following coblation, the wand was removed with the needle in place, and the motion segments were randomly assigned to three groups  were obtained from donors with no medical history of musculoskeletal disorders (UCSF Willed Body Program). From these spines, ten intact motion segments (2 motion segments per spine) were harvested and randomly assigned to two groups , dialyzed against diH2O  for 3 days, treated with activated charcoal  for 30 min, filtered (0.22 μm filter) and lyophilized.For alginate purification, sodium alginate  was dissolved in 1,000 ml ultrapure deionized water  overnight. Sodium periodate  was dissolved in 100 ml diH2O and added to separate alginate solutions to achieve different degrees of theoretical alginate oxidation  under stirring in the dark at room temperature for 24 h. The OAs were purified by dialysis against diH2O  for 3 days, treated with activated charcoal  for 30 min, filtered (0.22 μm filter) and lyophilized. The actual oxidation (4.2% ± 0.5% and 8.5% ± 0.3%) was determined by using the AmpliteTM Colorimetric Aldehyde Quantitation Kit (AAT Bioquest Inc.) according to the manufacturer’s instructions.Oxidized alginates (OA) were prepared by reacting sodium alginate  with sodium periodate (Sigma) using a previously described method . Brieflyex-situ pilot study  in an ot study , and basEnzymatic treatment or control buffer was delivered to the CEP under fluoroscopic guidance. Briefly, a 17G Tuohy needle  was inserted through the posterolateral annulus fibrosus and advanced to the central region of the superior CEP in each disc . After nA custom setup was used to evaluate the effects of CEP permeability enhancement on solute transport into the disc . To elim® aluminum foam disc, 7%–9% relative density, ERG Aerospace Corp, Oakland, CA) while submerged in 200 mL of fluorescein tracer solution   and open-cell porous aluminum foam blocks with similar porosity and stiffness of vertebral trabecular bone (Duocel in PBS) . A servo in PBS) , followe in PBS) .Immediately after the transport experiments, discs were snap frozen in liquid nitrogen to prevent any further fluorescein dispersion. Next, two 5-mm-thick-coronal slabs were cut with a low-speed diamond saw  . From eaNP tissue digests were assayed for fluorescence on a SpectraMax iD5 microplate reader , and the fluorescence readings  referenced to a standard curve of known fluorescein concentrations. Fluorescein concentrations inside the NP samples were calculated based on the measured water content from lyophilization and were compared between site-matched locations in the discs having collagenase-treated vs buffer-treated CEPs.CEP tissue digests were assayed for sGAG content using a dimethylmethylene blue assay (DMMB) . The digAliquots of CEP digests were also hydrolyzed in 6 N HCL for 18 h at 110°C. Neutralized acid hydrolysates were then assayed for total collagen content using a chloramine-T colorimetric assay , referenThe total sGAG and collagen contents in the CEP tissues were used to evaluate the effect of the enzymatic treatment on the CEP matrix composition.The axial force, crosshead displacement and time data were collected during the transport experiment and processed with a custom script  that calculates the disc modulus and the total creep strain. Disc modulus was considered as the slope of the stress-strain curve at maximum value of the stress in the linear part of the static compression phase. The total creep strain was considered as the ratio between total displacement at the end of the static compression phase and the initial height of the disc.via random intercept were used to estimate the least-squares mean fluorescein concentration at each NP location and the least-squares mean sGAG and collagen content in the CEP, and Tukey HSD post hoc tests were used to test for differences between groups. One-way ANOVA was used to compare the whole-disc compressive creep strain and modulus between groups. Statistical analyses were performed in JMP Pro 15; p < 0.05 was considered statistically significant.Mixed effects models accounting for multiple measurements per subject p < 0.0001) and 7-fold higher than discs with untreated CEPs . In discs with buffer-treated and sham CEPs, fluorescein concentrations were generally highest near the CEPs, at positions A  and D , and lowest in the center of the discs, at positions B  and C .Intradiscal enzymatic treatment of the CEP significantly improved solute transport into the bovine coccygeal discs . Specifip < 0.0001). In the discs with buffer-treated CEPs, fluorescein concentration varied with position; concentrations were generally highest adjacent to the CEPs, positions A (5.14 ± 3.54 μg/ml) and D (7.14 ± 2.26 μg/ml), and lowest in the center of the discs, at positions B (0.58 ± 0.18 μg/ml) and C (0.62 ± 0.19 μg/ml).Similarly, intradiscal enzymatic treatment of the CEP improved solute transport into human lumbar discs . The flup = 0.03; p = 0.01; Intradiscal enzymatic treatment significantly reduced the amount of solute-blocking matrix constituents in both bovine and human CEPs. Specifically, collagenase treatment reduced sGAG content, with treated bovine CEPs showing, on average, 40% lower sGAG content than their paired untreated CEPs  than their paired inferior CEPs  and 13.8% lower than the tissue NP samples at position B in the same discs (p = 0.026). Comparisons of mean sGAG between the NP tissue samples at the other positions were not significant.To understand how the enzymatic treatment delivered to the CEP affected the composition of the NP tissue, we measured the sGAG content in the human NP tissue samples that were assayed for fluorescein . CollageTo determine the biomechanical effects of CEP matrix modification at the whole-disc level, we compared the compressive modulus and total creep strain between groups by ANOVA. In the bovine discs, total creep strain and compressive modulus were similar between discs with the collagenase-treated CEPs, buffer-treated CEPs, and untreated CEPs . Likewisvia needle to the superior CEP under fluoroscopic guidance. Results show that collagenase treatment (0.15 U) significantly reduced the amount of sGAG in the treated CEP: enzymatic treatment of human lumbar and bovine coccygeal CEPs reduced sGAG by 33.5% and 40%, respectively  for matrix modification, as this enzyme can break down the peptide bonds in four types of collagens  (p on human CEP composition appeared similar to those observed with truncated human MMP-8 (The CEP is rich in type II collagen  and aggr and IV)  and has  and IV) . In the an MMP-8 , and togan MMP-8 . One posan MMP-8 .Targeted delivery of matrix-modifying enzymes to the CEP presents several technical challenges. We chose a 50 µl volume of injectate based on preliminary intradiscal pressure testing; injecting larger volumes increased the back pressure and made it difficult to prevent leakage. Choice of the hydrogel carrier was made to control the release of the enzymatic treatment , and intProper CEP function requires balancing conflicting biotransport and biomechanical demands, and thus, successful matrix modification strategies for improving biotransport must be sensitive to any biomechanical effects. Specifically, excessive cleavage and removal of collagen fibrils or proteoglycans in the CEP could inadvertently lower disc modulus and risk clinically significant disc injury. The collagen network helps resist transverse CEP stretching that arises during disc compression , and proThis study had several limitations. First, we removed the bony endplates before the transport experiments in order to isolate the effects of CEP matrix composition on solute transport into the discs and to facilitate comparison between species. For example, bovine spinal units have vertebral epiphyses and growth plates that persist through adulthood, whereas adult human discs do not. Also, the bony endplate is much thicker in bovine coccygeal vertebrae than human lumber vertebrae. Given the total permeability of the bony and cartilaginous endplates is more strongly correlated with CEP permeability than with bony endplate permeability , removinin vivo pre-clinical studies. Future studies should focus on optimizing enzyme delivery and release, as well as on discovering the native cells’ response to matrix-modifying enzymes and the by-products from matrix proteolysis.We found that intradiscal delivery of a matrix-modifying enzyme to the CEP significantly reduced the amount of sGAG. Treating the CEP in this manner dramatically enhanced small-solute transport into the disc. This is an important step towards Strategies for enhancing nutrient transport across the CEP may have the potential to slow or reverse disc degeneration. Our new results are significant because they suggest that intradiscal delivery of a matrix-modifying enzyme to the CEP can effectively reduce solute-blocking matrix constituents and thereby improve solute transport. This treatment strategy could stand alone or possibly be combined with biologic regenerative therapies that target the NP and increase nutrient demands.


The Farmers Market Nutrition Program (FMNP), a program of the U.S. Department of Agriculture (USDA), provides coupons to individuals/families enrolled in the Special Supplemental Nutrition program for Women, Infants, and Children (WIC) to purchase fresh produce from approved farmers markets. While some studies suggest FMNP may improve nutrition for WIC clients, there is limited research on program implementation in practice. A mixed-methods equitable evaluation framework was utilized to: (1) better understand the FMNP in practice at four WIC clinics on Chicago’s west and southwest sides that serve predominately Black and Latinx families; (2) detail facilitators and barriers to participation in the FMNP; and (3) describe the potential impact on nutrition. In this manuscript, we present qualitative findings from Aim 1. We identified six steps for how the FMNP has been implemented in our study setting and opportunities to improve the implementation of the program. Findings suggest that well-defined and consistent guidelines on: (1) how to seek state approval for farmers markets; and (2) coupon distribution and redemption, are necessary to optimize utilization. Future research should explore the impact of newly offered electronic coupons on redemption rates and fresh fruit and vegetable purchasing behaviors. The Farmers Market Nutrition Program (FMNP) (established in 1992) provides coupons to clients of the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) towards purchasing fruits and vegetables from approved farmers markets, farmers, and roadside stands. As an expansion of regular monthly WIC benefits, FMNP is intended to improve access to fresh fruits and vegetables, reduce food insecurity, and improve nutrition for pregnant and postpartum persons and their families. However, research is inconclusive regarding the extent to which FMNP coupons are redeemed, with some evidence that redemption rates are lower in inner-city communities than they are nationally ) and a community-engaged research arm (SUHI). Sinai Chicago serves the predominantly non-Hispanic Black and Latinx west and southwest sides of Chicago, areas of the city that have been historically disinvested and are disproportionately impacted by health and social inequities. Chicago is divided into seventy-seven officially recognized community areas . SCI hasA document review, semi-structured interviews, and a mobile survey were conducted to better understand the FMNP in practice. In July 2021, 16 foundational documents related to the Illinois WIC FMNP were received from SCI WIC sites and the Farm on Ogden. All documents were in English. Examples of documents include job descriptions of key staff and an FMNP informational brochure for WIC clients. A full list of documents can be found in the Criterion sampling was used to identify interview participants . From JuNvivo (released in March 2020) was used to conduct a thematic analysis of the client and staff interviews. First, four members of the research team reviewed the transcripts for recurrent codes. Two codebooks were developed: (1) a combined codebook for English and Spanish-speaking WIC clients, and (2) a codebook for the staff interviews (SCI-affiliated WIC clinic and urban farmers market). The researchers determined themes and sub-themes using the repetition of words, phrases, and meanings. The researchers then reviewed each other’s themes and discussed discrepancies to arrive at consensus. These methods allowed for a deeper understanding of the themes and relationships that emerged around decision making and regarding who receives FMNP coupons as well as practices across the WIC clinics. Finally, the researchers narrowed the themes and sub-themes based on the most common themes. The document review and semi-structured interviews revealed six key intended steps for the successful implementation of the FMNP. These six key components were examined within the context of the program in practice at the SCI-affiliated WIC clinics serving pregnant and postpartum persons on the southwest and west sides of Chicago. WIC clients, WIC staff, and farmers market staff supported the concept of the FMNP and recognized that the FMNP, in theory, could increase access to fruits and vegetables. Clients who redeemed the FMNP coupons in the current season reported a positive experience at the farmers market and described the importance of redeeming the coupons. Clients were able to supplement family meals with the fruits and vegetables that they were able to buy at the farmers market, as one client mentioned: I was a little short on money…I was trying to find a meal that was suitable for my pocket needs…I found different ingredients in my refrigerator, and there were a few items, like vegetables, that I was missing. And I remembered that I had those coupons…I sent my family member out to go get those vegetables, and it was useful because I didn’t have the total amount, so using the coupons helped me. I had just enough for what I needed [WIC Client 8].WIC staff noted that many clients who were aware of the coupons and had redeemed them in a previous season prioritized the FMNP coupons. Clients asked about the coupons as shared by a WIC staff member:There are participants…since kids can stay in the program until age 5, as long as they’re eligible for WIC, and they meet the criteria of farmers market coupons…that ask us every year… Hey, did the farmer’s market coupons come in? [WIC Staff 4].Clients who had not redeemed the coupons also believed that the farmers markets accepting coupons could provide high-quality, affordable produce compared to their local grocery stores. Some clients had past experiences at farmers markets while others shared that the perception of produce quality was a motivator to redeem the coupons as one WIC client explained:I went to [WIC to] get supplies for my daughter, and when they gave me the coupons, I saw that they were for fresh vegetables and fruit. And, I actually like that because I get to grab fruit that I like, and they’re fresh… [I know] they’re not sitting on the store for a while, not knowing if they’ve gone bad or [if] they’re gonna expire soon [WIC Client 7].Another noted,…I’m hoping that the food is fresher…The place that I was going to…I didn’t get the fruit because I used to get the fruit in the can because the open fruit (produce)… It didn’t look right. So, I’m hoping that it’s fresh fruit…Me and my kids, we eat a lot of fruit [WIC Client 5].Clients identified the WIC program as a long-standing, trusted social service program from which they can receive reliable nutrition information and education to support the wellbeing of their family as well as other resources they may not otherwise be able to afford. Clients reported they received information and education about healthy eating, cooking and recipes, breastfeeding, and child development. One client expressed how valuable it is that by participating in a nutrition lesson, she qualified for additional coupons which allowed her to obtain enough milk to last until her next appointment,Something that I didn’t know they was doing; they allow you to do a lesson online and then they give you a month worth of coupons. I’m running out of milk and stuff... So that’s definitely a plus [WIC Client 7].Another client shared how participating in WIC activities allowed her to earn and redeem points for resources to support safe sleep and active daily engagement for her baby.Yeah, [the WIC program] gave us a lot of stuff for my baby, like the points and stuff they was giving me points. And the points build up to get like a playpen, and a bassinet, I have got a lot of stuff from them...toys for my baby [WIC Client 9].Overall, both WIC clients and staff provided positive feedback about the FMNP. However, they also identified potential opportunities to improve program processes and further develop the partnership between communities, organizations, and health centers, which focused on further improvements to accessing fresh fruits and vegetables. Some WIC clients who received FMNP coupons, but did not redeem them, indicated they were unsure if they were provided information about which farmers market locations accepted coupons. As noted by one of those clients: I told [staff member] I didn’t know where [a farmers market] was at and [the staff member] said I could look it up...said I could look it up on the internet [WIC Client 7].To ensure anyone who could benefit from coupons has an opportunity to receive them, one client suggested ensuring that processes are standard across all WIC sites:…just making sure that every counselor is specifically telling every participant like, “Hey, you could go to a farmer’s market,” would definitely be beneficial, and probably improve the program so the people are aware of that benefit [WIC Client 15].Other clients suggested developing partnerships to raise awareness of the FMNP such as organizing smaller pop-up shops in communities where there may not be farmers markets. The FMNP is a federally funded program intended to improve access to fruits and vegetables for pregnant and postpartum persons and those with young children in low-income communities. The findings of our study demonstrate the complexity of the real-world implementation of the program in practice on the west and south sides of Chicago and neighboring suburbs. We identified six key components required for implementation of the FMNP: (1) farmers markets seek authorization to accept FMNP coupons; (2) WIC staff receive training on the FMNP program; (3) USDA sends FMNP coupons to participating WIC sites; (4) WIC staff notify clients of FMNP program and assess client nutrition needs to determine coupon issuance; (5) WIC clients identify participating farmers markets and shop for preferred foods; and (6) farmers markets stamp and collect coupons. Having knowledge of how the FMNP operates and documenting these steps has the potential to make the authorization process easier for farmers markets, optimize FMNP coupon distribution within the WIC clinic setting, and ultimately increase access to farmers markets and therefore fresh fruits and vegetables for WIC clients. In order to encourage full participation and maximize benefits for clients, state-specific guidance documenting the program flow should be communicated clearly and accessibly. It is worth noting that we found that WIC staff and clients believe the FMNP has the potential to markedly improve access to fresh produce and reduce nutritional inequities. In line with existing research, clients who redeemed their coupons were able to improve the variety and nutritional composition of home-cooked meals with farmers market purchases , purchasThis study allowed us to fully describe the FMNP in practice from both the perspective of WIC clients who received FMNP coupons, as well as farmers market and WIC staff. We identified six steps for how the FMNP has been implemented in Chicago, as well as opportunities to improve the implementation of the program, including standardizing processes across participating FMNP-distributing WIC clinics and farmers markets, and building upon the trusted relationships many WIC clinics have within their communities. In order to address food insecurity in urban communities such as Chicago, it is important to better understand the programs that exist to reduce these inequities. A notable strength of this study was our partnership with the community, participating WIC clinics, and an urban farm on the west side of Chicago. Working with these partners allowed us to refine data collection tools, develop recruitment tactics, and interpret our findings within the context of the community of interest. Similarly, our use of equitable evaluation principles to guide this evaluation presents a model for how other studies can partner with stakeholders to conduct evaluations that are grounded in cultural humility and acknowledge historical barriers to equity. Finally, it is important to note that this study was conducted during 2021 when monthly WIC cash-value benefits had increased in response to the COVID-19 public health emergency. As a result, WIC clients were receiving a larger benefit compared to pre-pandemic, but were still interested in additional avenues to access fresh produce.This study was conducted at the height of the COVID-19 pandemic. As such, there were logistic challenges with the implementation of the program and with data collection. WIC clients were able to meet with their WIC counselors virtually, making it challenging to distribute physical coupons to clients. Similarly, recruitment for our evaluation initially took place in person at WIC clinic sites, and therefore limited research subjects to those who were still coming on site for services. In order to increase our sample, we expanded recruitment methods to using snowball sampling, social media recruitment strategies, and word of mouth with partners. Another limitation was the shortened period for which WIC clients were able to use their coupons. In the 2022 season, participating SCI-WIC clinics did not receive this season’s FMNP coupons for distribution until September. Coupons are only accepted until October, greatly reducing the time period in which clients could utilize them. Finally, our study was not designed to compare outcomes for pregnant versus postpartum persons. Future research should explore the potential barriers that may arise for each of these stages of pregnancy. The present study identified six steps for the real-world implementation of the FMNP in an urban setting. Findings suggest that key stakeholders see significant value in the program, but provided recommendations for improvements in the distribution of coupons, communication about the program, and access to farmers markets. First, there should be clear guidelines and information on the FMNP, including the location of FMNP state-approved farmers markets, hours of the markets, information on available transportation to farmers markets, and product availability. Next, it is important that the coupon season be long enough to provide adequate time for clients to use coupons. Delay in coupon distribution severely limits client ability to purchase foods from farmers markets. It is critical to document clear steps for all actors involved in the program to ensure equitable access and distribution of coupons. States are now able to submit a proposal to the USDA Food and Nutrition Service (FNS) requesting the use of electronic solutions to distribute FMNP coupons instead of using traditional paper coupons . This mo