In 2006, the United States Agency for International Development established the Neglected Tropical Disease (NTD) Control Program to facilitate integration of national programs targeting elimination or control of lymphatic filariasis, onchocerciasis, schistosomiasis, soil-transmitted helminthiasis and blinding trachoma. By the end of year 3, 12 countries were supported by this program that focused first on disease mapping where needed, and then on initiating or expanding disease-specific programs in a coordinated/integrated fashion. The number of persons reached each year increased progressively, with a cumulative total during the first three years of 98 million persons receiving 222 million treatments with donated drugs valued at more than $1.4 billion. Geographic coverage increased substantially for all these infections, and the program has supported training of more than 220,000 persons to implement the programs. This current experience of the NTD Control Program demonstrates clearly that an integrated approach to control or eliminate these five neglected diseases can be effective at full national scale. The neglected tropical diseases (NTDs) are a group of conditions causing significant morbidity and mortality worldwide but which until recently received only minimal attention from most of the world, largely because they affect the poorest, most vulnerable and most disenfranchised members of society.1Among the 15 most prominent NTDs,Most of the drugs used in these single-dose, once- or twice a year treatment regimens are donated through large public-private partnerships that bring together public health implementers, public-sector and private-sector donors, and the major pharmaceutical firms producing these drugs .4Loa loa co-endemicity with onchocerciasis or LF).3Historically, many Ministries of Health in disease-endemic countries have supported the control of NTDs through independent, often parallel, programs, with each maintaining its own planning, funding, drug supply chain, MDA campaign, monitoring, and evaluation. If funding were available for one program, that program might have been able to implement preventive chemotherapy while a sister program could not. However, because there is considerable overlap of these diseases in persons and communities, controlling one of the NTDs and not others that could be managed through a similar strategy is inefficient at best. Furthermore, research has provided sufficient evidence to suggest that co-implementation of the integrated PCT programs is safe for persons and communities in all but a few specific settings  to support and empower national governments to develop integrated NTD control programs embedded, where possible, within existing service delivery platforms and to lead these programs in scaling-up activities to full national levels; 2) to provide technical assistance for planning, budgeting, reporting, and complying with international standards and guidelines  to improThe prime contractor, RTI International, has provided grants and coordination for a team of non-governmental organizations (NGOs) and implementing partners to support integrated NTD control programs organized and led by the governments of selected countries. The support from the U.S. Government was intended to build on existing commitments by governments and other donors and fill financial and technical gaps that were preventing national programs from reaching full national scale. The program was mandated to track and report on the additional numbers of persons reached and treatments provided through support of the NTD Control Program .The countries currently involved in the USAID-supported, RTI-coordinated NTD Control Program are identified in All drugs were distributed by the national Ministries of Health whose national NTD control programs determined how best to implement their MDAs . The druAlbendazole for treatment of LF was donated by GlaxoSmithKlineBecause knowledge of the distribution of each NTD in a country is absolutely essential for developing any implementation (or integrated-implementation) plan, disease-specific mapping was carried out according to guidelines recommended by WHO and its partners . AlthougTo document progress toward achieving the program's targets of 40 million additional persons treated with 160 million treatments over 5 years, the following indicators were used to track country-specific program progress: 1) number of countries supported by the NTD Control Program; 2) number of additional districts (implementation units) mapped for each endemic disease; 3) number of people treated  and recorded in MDA registers for each round of PCT; 4) number of additional  treatments provided (i.e. number of times a single drug dose is administered) and recorded in MDA registers for each round of PCT; 5) number of additional implementation units reached and reported by national programs for each round of PCT; 6) programmatic coverage: % targeted population reached with appropriate PCT treatment each round of PCT and calculated from register reports as the number treated divided by treatment-eligible population targeted in the implementation unit (as defined by census reports); 7) geographic coverage: % of endemic districts covered by PCT programs; and 8) number of persons trained for integrated NTD control through support from the NTD Control Program.Funding levels for each of the program's first three years approximated $13.5 million per year, with the mandate that at least 80% of total annual resources be allocated to country program implementation and that overall management costs by RTI  be no more than 20%. Of the 80% allocated for country implementation activities, approximately 20% was earmarked for procurement of essential drugs not available through donation programs .Because knowing the distribution of the targeted NTDs is essential for developing an implementation strategy, the first efforts of the NTD Control Program in the participating countries focused on cataloging the disease-distribution information available and then supporting on-the-ground efforts to map the distribution of infection where sufficient information was not available.Although programs targeting the individual NTDs were active in many countries prior to the inception of the NTD Control Program, few countries approached these diseases with an integrated control strategy. As seen in Because a treated person in these integrated programs often receives a drug package comprising more than one medication, the metric ‘treatments provided' was developed to record the number of individual drug treatments received by the target population. As indicated in As the national programs supported by NTD Control Program funds increased in number and expanded in scope, the number of donated tablets of drug delivered to the national NTD programs has also increased. sine qua non for success of preventive chemotherapy programs is high rates of drug coverage in the disease-endemic populations. Although varying among countries and for specific programs within each country, programmatic coverage  was generally good . At the heart of all PCT programs is the community that is affected by NTDs. Training is designed to empower these communities to treat NTDs within their own populations. The NTD Control Program has supported the training of more than 220,000 persons during its first three years, with most being community-based health workers/drug distributors . WorkingAnalysis of expenditures during the first three years shows that the program fulfilled its mandate that at least 80% of program funds be spent on country program implementation. During the first three years, the program received $40,728,320 in funding. Program expenditures by the end of year 3 were $37.9 ($2.8 million remaining to be spent), with 81.3% expended for country program activities and purchase of essential drugs and with 18.7% used for overall management of the program .The breakdown of program expenditures reflects the diverse set of activities that must be supported to enable treatment of persons at the community level. Of the 81.3% of funds going directly for country program activities and supplies , the larAlthough not yet specifically documented for this program, it should be noted that previous costing studies of programs to control one or more NTDs have shown that in addition to the external support received by disease-endemic countries, the national ministries contribute approximately an equal level of resources in staff salaries and in-kind resources to achieve success.14The whole concept of program integration is, undeniably, complex and involves, as described by some,Early pilot studies confirmed the general feasibility of successful integration of NTD control efforts,From the data in Introduction of this funding also effected a major qualitative change in national programs targeting the NTDs. Even though in the year prior to the NTD Control Program, up to 33 million people had received treatment of NTDs in disease-specific programs in the target countries, these national programs were constantly challenged to identify funds, most often on a yearly basis, to support their program activities. With the NTD Control Program, a secure funding source was established, so that these national programs not only could achieve broader disease control in their populations but also could undertake proper planning to address their NTD problems more effectively and cost efficiently.Different from single-disease programs where each person reached equals one treatment given, in the integrated programs each person reached receives a drug ‘package' usually containing more than one drug. Therefore, the metric ‘treatments delivered,' the number of times a single drug dose is administered, had to be developed to record this programmatic achievement. More than 222 million treatments were provided by national programs during the first three years of the NTD Control Program. It can be predicted with certainty, based on recent studies assessing the impact of disease-specific NTD programs,What even these numbers by themselves fail to impart, however, is an understanding of the magnitude and importance of the public-private partnership between specific pharmaceutical companies and the public sector. It has been the donation of extraordinary amounts of drugs  that has made possible the successes of these global efforts to control or eliminate the NTDs. In addition, although the drug manufacturers have without question provided most of the drugs used in this program, the generic drugs needed for schistosomiasis and STH control and for LF elimination in countries outside Africa have also been obtained and provided to national programs in the NTD Control Program by a number of governmental and nongovernmental partner organizations, including the Department for International Development (United Kingdom), the United Nations Children's Fund, University of Notre Dame, WHO, the World Food Program, and World Vision.In addition to its role in facilitating integrated national MDA programs to provide treatments for their at-risk populations, the NTD Control Program has, of necessity, supported countries in mapping the distribution of infection and developing, in collaboration with WHO, action plans that will enable them to begin or expand implementation of drug delivery in the coming years. Already the empirical experience gained from three years of program activities in the first seven implementing countries has contributed appreciably to the development of new guidelines, norms, and regionalized strategies that will facilitate and accelerate program activities in the many countries still needing to begin their integrated NTD control programs.Major challenges still lie ahead for creating integrated programs targeting NTD control or elimination, not the least of which is the large number of countries requiring these programs. Such a need provides political challenges not just for the national governments, which must commit their limited resources and energy to the programs, but also for the bilateral donors to support these national commitments, the drug manufacturers to sustain their long-term pledges, and the various implementing partners necessary to support national programs to carry out their integrated NTD implementation strategies.In addition to these political challenges, technical challenges also remain, the first being definition of the geographic scope of each of the NTDs. Mapping the infections must be completed, not just to identify prevalence, but, even more importantly, to define exactly what action must be taken at which level of prevalence and in which geographic area for each of the NTDs. Then, once it is determined what action is required for each of the NTDs, the treatment gaps for specific diseases  must be19In just three years, the NTD Control Program has exceeded program expectations that had been based on previous pilot program activities. This achievement demonstrates not only the feasibility of going to national scale with an integrated approach but also that efficiencies can be achieved through integration. These efficiencies come, in large part, from removing duplication in the operations of specialized disease programs as co-implementation is introduced. They also stem from the ability of the NTD programs to implement PCT through community networks, schools and existing health service delivery platforms, such as child health days. Going forward, even greater gains may accrue as opportunities are pursued strategically to leverage or complement other development-sector efforts, such as water and sanitation improvements, or other health-sector inputs, such as malaria control activities. This type of integration of NTD control within development efforts and on existing health-service delivery platforms holds promise for even greater program efficiency and its positive impact on strengthening national health systems. Although it is clear that not all integrated NTD program activities will have such health system strengthening effects,21Although the creation of integrated NTD control programs has brought with it a wide range of political and technical challenges whose importance should not be underestimated,


Despite its great public health importance, few control initiatives addressing podoconiosis  exist. In June 2010, the first podoconiosis program in Northern Ethiopia, consisting of prevention, awareness, and care and support activities, began in Debre Markos, Northern Ethiopia. This study aims to document and disseminate the lessons learned from a new community podoconiosis program in Debre Markos.We used a content analysis approach to examine and evaluate data from a series of sources. These sources include conducted interview transcripts, a focus group discussion transcript and secondary sources including monitoring and evaluation field reports, observation notes, and research obtained from a literature review. Themes were identified and grouped into matrix tables. Overall, sixteen program steps were identified and grouped into 6 domains: Initial preparation, training and sensitization, foundation building, treatment activity implementation, awareness, and follow-up. Emphasis is placed on the need for baseline data, effective training, local leadership, experience-sharing, mass-awareness, cross-cutting sector issues , and integration with government health systems. Related successes and challenges are also described, as are stakeholder roles and misconceptions and socio-cultural challenges affecting the program start-up. Many of the identified successes and challenges are relevant to the aim of the podoconiosis program to be sustainable and community-led.Much of this information has already been used to improve the Debre Markos program. We also anticipate that the domains and steps identified will be useful in guiding new programs in other settings where podoconiosis is highly prevalent. We hope to encourage partnerships and collaboration among podoconiosis stakeholders in future growth and disease control expansion. Podoconiosis is a chronic non-infectious disease that causes leg swelling among those living and walking bare-footed in red clay soil areas. It can be prevented and treated primarily by the use of shoes and foot hygiene. In Ethiopia, it is estimated that nearly 11 million people are at risk but few control programs exist. We aimed to assess and document the lessons learned from the first community podoconiosis program started in Northern Ethiopia in June 2010. We conducted interviews and a focus group discussion in addition to examining monitoring and evaluation field reports, observation notes, and other research articles. Overall, sixteen program steps were identified and grouped into 6 domains: Initial preparation, training and sensitization, foundation building, treatment activity implementation, awareness, and follow-up. Related successes and challenges, stakeholder roles, misconceptions and socio-cultural challenges affecting the program start-up were also identified. We hope that the results will be useful in guiding new programs in other settings where podoconiosis is highly prevalent. Podoconiosis, or non-filarial elephantiasis, is a geochemical type of neglected tropical disease that affects individuals, often barefoot subsistence farmers, exposed to the red clay soil originating from volcanic rock Podoconiosis is staged using an adapted Dreyer staging system for lymphatic filariasis. This adapted system has five stages based on the proximal spread of swelling, knobs and bumps in addition to the measurement of moss presence (M+ or M−) and the below-knee circumference. Podoconiosis can be distinguished from filarial elephantiasis through history and clinical examination: podoconiosis develops first in the foot, it causes bilateral but asymmetric swelling often confined to the lower leg, and groin involvement is rare in podoconiosis. In contrast, lymphatic filariasis can extend above the knee and has the potential for groin involvement. Another common differential diagnosis apart from podoconiosis is leprosy lymphedema. Patients with podoconiosis can be distinguished from leprosy lymphedema because sensation persists in the toes and forefoot, and trophic ulcers, thickened nerves and hand involvement are not experienced Podoconiosis is of public health importance in highland zones of Africa Despite the public health importance and prevalence of podoconiosis, few control initiatives exist. In Wolayta, Southern Ethiopia, the Mossy Foot Treatment and Prevention Association (MFTPA) has been recognized as a successful ongoing podoconiosis community program model since 1998 as compared to the World Health Organization (WHO) Innovative Care for Chronic Conditions (ICCC) Framework This evaluation was performed to provide feedback to the Charities & Societies Agency, local stakeholders and funders, as required by non-government organizations under Ethiopian law. The evaluation was not submitted to an IRB or ethical committee review because information was gathered with the above purpose in mind, and no additional data including individual patient medical records were gathered purely for research purposes. Oral consent was obtained from participants prior to interviews and documented in a participant checklist. Oral consent was used instead of written consent because a portion of the participants were illiterate.The study was conducted in East Gojam Zone, Amhara Region, Northern Ethiopia. The zonal capital is Debre Markos, a town located 300 km northwest of the capital city Addis Ababa at an altitude of 2446 meters in the highlands above the Blue Nile Gorge. The main spoken language is Amharic This study focused on the first year of the new Debre Markos community podoconiosis program see . This prData were collected through a series of eight semi-structured interviews  and one focus group discussion . The interviewees were selected to represent stakeholders involved in the set-up of the program, including community members, health professionals, governmental officials, church leaders, and program staff. Stakeholders can be defined as “actors who have an interest in the issue under consideration, who are affected by the issue, or who because of their position have or could have an active or passive influence on the decision-making and implementation process,” A content analysis approach was used to examine the interview transcripts, the FGD transcript, the monitoring and evaluation field reports, observation notes, and literature. Each document was examined jointly by two of the investigators multiple times. In order to minimize interviewer biases, the interview/FGD transcripts were made anonymous during analysis. Phrases relating to program start-up steps, stakeholder roles, and misconceptions and socio-cultural challenges affecting the new program were identified and grouped into themes and matrix tables. These themes were continuously reviewed and changed as new ideas arose to ensure completeness.–From the data collected, sixteen program steps were identified including relevant program successes and challenges. These steps were grouped into 6 domains: Initial preparation, training and sensitization, foundation building, treatment activities implementation, awareness, and follow-up. As seen in Additional notes, successes, and challenges are documented for each step. Emphasis is placed on the need to gather baseline data, for effective training, local leadership, experience-sharing, mass-awareness, cross-cutting sector issues (i.e. water and waste management), and integration with government health systems. Related successes and challenges were also identified, many of which are relevant to the aim of the podoconiosis program to be sustainable and community-led..Across these program steps, many different stakeholders were found to hold a role, each being significant to the success of the activities. Stakeholder roles in establishing the new program were identified as shown in A range of misconceptions about the disease and about wearing shoes were reported by community members and affected individuals, many of which are likely to influence the program in its early stages.A wide range of beliefs about podoconiosis were expressed, ranging from physical to spiritual –“Some people believe that podoconiosis is caused from standing up too fast or the hot soil burning their skin” “The community doesn't believe that the cause of podoconiosis comes from the soil. More people think that it comes from contact with blood, evil spirits, or a curse from God” “Many relate podoconiosis to magic” “Some believe podoconiosis is caused by wearing bad shoes or acts of sin by individuals or families” “We thought podoconiosis was caused by evil spirits, family links, sunlight, or touching the soil around cemetery graves “Some of us just accepted that we had to live with an unknown condition with an unknown cause” Others think podoconiosis comes from contact with animal blood. These beliefs affect the program because it takes time to convince patients otherwise” “Many believe that shoes are a sign of laziness. The sign of strength is going without shoes. Affected individuals and community members also reported many challenges faced by people with podoconiosis in daily life. Several of these are likely to have direct impact on attendance at a newly launched program.“Many persons wade in the river to be able to “wash” their feet but the river is thick with silica soil and it doesn't properly clean their feet. Others are too tired to wash their feet after a long day working in the fields.” “Religious practices affect the hygiene necessary to prevent podoconiosis. For example, after communion, Orthodox lay persons often don't wash their body for one day. Once affected, podoconiosis patients then can become excommunicated because of stigma. Women affected by podoconiosis may not be able to marry” “Everyone stigmatizes patients because they are not with God” “Patients mostly think that if they are seen going for treatment, they will be further stigmatized and they don't believe there is hope” “Many of us patients aren't registered because it is too difficult to travel from remote areas” “Patients' use of traditional medicine and inability to buy shoes in addition to a lack of attention by clinical health workers negatively affect treatment health programs” The results gathered are intended to report the lessons learned from the start-up of the Debre Markos podoconiosis program, the first of its kind in Northern Ethiopia. From these results, two important lessons learned can be noted as critical components to effective program start-up:the importance of government engagement through a training workshop and joint development of a strategic plan, andthe need for early community events and engagement aimed at diminishing stigma and fatalism, enabling patients to identify themselves and come forward for treatment.The focus group discussion of patients with podoconiosis added particular value to the results presented. Their experiences approaching and navigating the podoconiosis treatment for the first time and their experiences with associated stigma provided a unique perspective that also dictates local podoconiosis program start-up steps. In addition, the misconceptions concerning podoconiosis and the socio-cultural challenges reported by people with podoconiosis were likely to influence a new program.“These beliefs affect the program because it takes time to convince patients otherwise” It was our experience that these misconceptions act as barriers to new programs because patients need to be educated on the true cause of disease and convinced of the value of the hygiene treatment practices suggested. The practical clinical improvement seen by patients was a powerful tool in overcoming these misconceptions. After witnessing the improvement of others with the disease, a majority of patients turned away from their earlier misconceptions about the cause of podoconiosis and were more willing to follow the recommended hygiene treatment. The daily challenges faced by people with podoconiosis also create additional program barriers that need to be addressed in a sensitive manner.Overall, we hope the results of this study are a fair and honest representation of the perceptions, challenges, and successes surrounding the beginning of the Debre Markos program and its ability to address podoconiosis. However, several limitations should be noted in this study's approach.An “insider-outsider” perspective was taken to conduct the study, meaning that the authors were involved in the events described in the study This “insider-outsider” perspective captured a cross-section of stakeholders. It could have been colored by changing positions, events, and context. The study was conducted as an evaluation to provide feedback to the Charities & Societies Agency and local stakeholders and was thus, resource-constrained. If further resources had been available, the evaluation could have been made more comprehensive by including an expanded cross-section of stakeholders and additional community participatory methods. Particular stakeholders such as those from the education sector were not included in the interviews. As noted in It should also be recognized that other groups such as the Mossy Foot Treatment and Prevention Association (MFTPA) in Wolayta, Southern Ethiopia have important experience to contribute to other podoconiosis programs. Against the WHO ICCC Framework, it was found that their strength at the micro-level is evident particularly through involvement of treated patients as community patient agents (CPAs) and through network groups composed of local leaders This study was conducted to document and disseminate the lessons learned from the experience of starting up a community-based prevention program. These lessons are illustrated in the form of project process steps, related project challenges and successes, stakeholder roles, and misconceptions and socio-cultural factors affecting program start-up. The results of the analysis have been used to improve the design and implementation of the present Debre Markos program and increase its chances of sustainability. Logical frameworks have been developed and assumptions have been identified. This qualitative information could hold more depth than quantitative survey research in designing strategies for change.We anticipate that these lessons will be useful in guiding future programs in other settings where podoconiosis is highly prevalent. In the future, a comprehensive regional and national podoconiosis prevalence mapping is necessary to identify endemic areas with high disease burden. Associated patient-led groups should be established in more remote areas identified to further the reach of the treatment site and an effective patient ‘graduation’ scheme needs development. Government ownership must continue to be promoted at all national, regional, and local levels. Podoconiosis activities should be mainstreamed through the appropriate government directives  and decentralized community initiatives. Specific pre-service and in-service training for health professionals at every level is necessary. Partnerships with local government bodies and other existing NGOs must be encouraged and podoconiosis prevention and treatment activities integrated with existing school health education programs and community water projects. Currently, there are school health education programs implemented in Amhara Region that focus on hygiene education through peer-group leadership and creative activities such as music and drama, which might be used for podoconiosis education. The following domain steps might also provide opportunities for integration with other neglected tropical diseases: Domain One. Initial Preparation/Step C. Approach the local government and community leaders in the identified area (government ownership); Domain Two. Training and Sensitization/Step A. Organize a one day program sensitization workshop and Domain Five. Awareness/Step A. Develop Information Education Communication/Behavioural Change Communication materials and Domain Six. Follow-up/Step A. Follow-up with patients at household level. There is a common hygiene message with lymphatic filariasis , trachoma, soil-transmitted helminthes and a common shoe use message with soil-transmitted helminthes With this study, we also hope to encourage partnerships and collaboration among podoconiosis stakeholders. The potential for experience-sharing and collaboration among podoconiosis stakeholders has the power to arouse public interest and stimulate action in addressing this neglected tropical disease.Figure S1Debre Markos podoconiosis program (began 2010) logframe with vision, goal, activities, and monthly measurable indicators.(DOC)Click here for additional data file.


N = 4.167). Results showed that the respondents had good and positive perceptions of the training program and found it very valuable, particularly with respect to training instructors and familiarization with the project. Besides, the training participants were able to acquire attitude, knowledge and skills that are conducive to the successful implementation of the program. Based on the subjective outcome evaluation findings, it is concluded that the training program was effective in helping the participants to acquire the necessary knowledge, attitudes and skills in implementing the program.Subjective outcome evaluation findings based on the perspective of the participants participating in a 3-day training program of the Project P.A.T.H.S. are reported in this paper. The findings were based on the data collected from the training workshops conducted between 2005 and 2009 ( To promote positive and holistic development of adolescents in Hong Kong, The Hong Kong Jockey Club Charities Trust provided an earmarked grant of HK$ 750 million (HK$ 400 M for the first cycle and HK$ 350 M for the second cycle) to initiate and support the development, implementation, and evaluation of a positive youth development program entitled “P.A.T.H.S. to Adulthood: A Jockey Club Youth Enhancement Scheme” for junior secondary school students. The Project P.A.T.H.S. is a ground-breaking, indigenously developed, multiyear positive youth development program in Chinese context. There are two tiers of the programs, with the aims to promote holistic development of adolescents by providing opportunities and recognition for them to develop competence and skills which are conducive to positive youth development, promoting bonding with others, and holding healthy beliefs and clear values. The Tier 1 Program is a multi-year universal program designed for all Secondary 1 to Secondary 3 students based on 15 positive youth development constructs identified from the existing successful positive youth development programs , while tIn different fields, substantial time, money, man power, and energy are usually invested in in-service training programs. It is generally believed that investments in in-service training are associated with individual and organization benefits. In the same vein, in-service teacher training is commonly seen as one of the means to help teachers continue to acquire and enhance their professional knowledge and skills –6. In adIn Hong Kong, there are various in-service training programs offered for teachers and social workers, which are mainly organized and conducted by the Education Bureau, Social Welfare Department, and other nongovernmental organizations. Like many practitioners in other fields, teachers and social workers may be unenthusiastic to adopt new programs or practices  unless they feel secure and confident that they can make them work. Frontline experiences show that teachers' confrontation might hamper the implementation of a new program. It is believed that the provision of pre-implementation training can increase the confidence of the program implementers. Veenman et al.  suggesteSystematic and training programs were designed and offered to all participating schools in the P.A.T.H.S. Project. The program implementers  involved in the Project were invited to participate in a 3-day training workshop, consisting of 20 hours of training for each grade (Secondary 1 to Secondary 3). Each training workshop provided 12 sessions of training, which were held in 3 days within the same week. The 3-level progressive training program has six general objectives which to help the potential program implementers: (a) to understand the nature of adolescent development and the related issues and to cultivate a positive attitude to adolescent development; (b) to understand the nature of positive youth development, including its basic concepts, related programs, and research; (c) to familiarize themselves with the nature of Project P.A.T.H.S., including its basic philosophy, design, implementation, and evaluation; (d) to understand the content of the Tier 1 Program, including the core program and elective program; (e) to acquire the attitude, knowledge and skills that are conducive to the successful implementation of the Tier 1 Program; (f) to establish a self-help support network among the program participants . The sixAccording to Shek and Chak , the traTo appraise and determine the effectiveness of the training programs, program evaluation is indispensable. Scriven  defined Definitely, the training participants must be engaged in the evaluation process and their voices should be heard. As stated by the Joint Committee on Standards for Educational Evaluation , stakehoLevel I: reaction is a measure of participants' perceptions to the training. It is basically a measure of client satisfaction.Level II: learning is a measure of what the participants have learnt and gained from the training.Level III: transfer is a measure of changes in their behavior when they return to the job after the training program. It also refers to the transfer of training, which measures knowledge and skills gained in the training that are applied on the job.Level IV: results are a measure of the final outcomes and the change in productivity that occur owing to the contribution of the training.Although a number of criticism –20 and mA total of 121 workshops were conducted between 2005 and 2009 , with 4,The 31 items of the questionnaire were used to assess the participants' satisfaction with the training program, the instructors, as well as their views towards their own performance. There are several sections in the subjective outcome evaluation questionnaire outlined as follows:participants' basic demographic information,participants' perceptions of the training program, including the program objectives, design, activities format, and interaction among the participants (16 items),participants' perceptions of the instructors, including the understanding of the course, teaching skills, and professional attitude (5 items),participants' perceptions of their own performance, including involvement during program, application of their learning, and having confidence in the project implementation (4 items),participants' perceptions of the administrative arrangement, such as program enrolment, hospitality, venue, and facilities (6 items),things that the participants appreciated most (open-ended question),aspects of the program that require improvement (open-ended question).The questionnaire consisted of 31 items with a six-point Likert scale (1 strongly disagree to 6 strongly agree). Reliability analysis shows that the internal consistency of the whole scale was good . Besides, all subscales on the training program (16 items), instructors (5 items), participants' own performance (4 items), and administrative arrangement (6 items) were also reliable. The alpha values, mean interitem correlation are presented in n = 2,863) were female and 31.2% were male , with 5 participants not disclosing their gender. In addition, 72.2% of the participants  were teachers while 26.1% of the participants  were social workers. There were 66 participants from other disciplines such as teaching assistants or program workers. The mean years of the self-reported work experience were 10.5 years, from a minimum of 0 to a maximum of 45 years .The mean number of participants per workshop was 39.5. The total number of evaluation questionnaires completed was 4,167 . Among tN = 4,040), strengthening of the participants' understanding of positive youth development , encouragement of instructors to do their best , promotion of the participants' understanding of the Project P.A.T.H.S. including its basic philosophy, design, implementation, and evaluation , enhancement of participants' understanding of the Tier 1 Program , and strengthening of the participants' understanding of the nature of adolescent development . In particular, 97.3% of the respondents  cherished the peer-interaction amongst participants and 93.3% of the respondents  perceived that other participants were satisfied with the training program as well. Several observations can be highlighted from the quantitative findings based on the closed-ended questions (31 items). First, the participants generally had a very positive perception of the program contents and activities format of the training program , includiN = 3,986) perceived the instructors in a positive and encouraging manner  of the respondents thought that the instructors showed good and professional attitudes. 96.1%  of respondents perceived that the instructors had good mastery of the training curricula and that their teaching was clear and easy to understand . Third, regarding the performance of the program participants during the training process, a high proportion of the respondents had positive evaluation of their own performance  in the training program . In addition, they reflected that they are willing to apply the specific skills and theories learnt from the training program  and having confidence in program implementation after attending the training program . Finally, as shown in N = 4,098) and the reception provided by the training team . The participants also appreciated the workshop assigned  and the venue facilities .Second, as indicated in g manner . About 9 program . For insThe subjective outcome evaluation findings based on the responses of the potential program implementers in the Project P.A.T.H.S. are presented in this paper. According to the evaluation results, the current outcome evidently indicated that the training program had been successfully implemented and the program outcomes were very encouraging. With reference to Kirkpatrick's four-level model, the present quantitative findings showed that the training programs of the project P.A.T.H.S. generally generated positive reactions (Level I), enhanced learning (Level II), and desired behavioral changes (Level III). Several observations can be highlighted from the present findings. First, in relation to reaction data (Level I), favorable reactions were found. The training program was well received by the training participants. A large majority of participants reported that the program met their expectations. For instance, the overwhelming majority were satisfied with the training program, its content and format, training instructors, and the overall administration arrangement. As such, it is considered that the first four objectives of the training program have been achieved. In fact, the judgments about the worth and merit of the training program are very critical. If the program implementers were not satisfied with the training experience, they might not buy-in the Project P.A.T.H.S. and would not use what they have learnt in their teaching. They may even advise their colleagues not to buy-in the project and join the training program. Consequently, the well-designed training programs contributed and facilitated the development of positive attitudes among the program implementers towards the P.A.T.H.S. Project. With reference to the domain of learning (Level II), the results provide reliable and constructive evidence. From the perspectives of the participants, there is no doubt that the training was tremendously valuable. For example, most of the participants indicated that the program provided sufficient information about the Project, which can help them cultivate their understanding of the nature of youth and positive attitude to adolescent development. In addition, almost all the respondents perceived that the training program strengthened their understanding of positive youth development and Project P.A.T.H.S., including its philosophy, program design, implementation and research. All these positive feedbacks suggest that learning effects were found in the training program. It is believed that the participants could gain knowledge or increase their awareness of the new program implementation. The P.A.T.H.S. training therefore seems to have fulfilled its fifth objectives. In particular, the findings revealed that the knowledgeable and skillful facilitation of the training instructors was highly appreciated and valued. As such, the program implementers could gain an understanding of the humanistic approach to teaching and actual implementation of the Project P.A.T.H.S. though the train-the-trainer workshops. Furthermore, a large majority appreciated the peer-interaction amongst participants. The training workshops actually help them to establish a self-help support network; the interactions among program participants and with the instructor are crucial for social learning. It is believed that the training of Project P.A.T.H.S. provided a supportive environment for the participants to grow and to learn from one another through interaction with other members. This addressed the sixth objective of the training program.Concerning the behavior and performance change in teacher practice (Level III), the impact should be evaluated after a period of time to ensure the change has occurred. Based on the quantitative findings, it is shown that there were positive influences on the beliefs and attitudes of the participants, and the results suggest that the confidence of the program implementers was boosted. For instance, a high proportion of participants indicated that they are willing to apply the specific skills and theories learnt from the training program. The positive intention of integrating newly learnt knowledge and skills into their practice suggests the support of training transfer and behavior change, and thus students will then benefit from teacher practices. Of course, post-training and implementation research in this context is important.As indicated in This study was aimed to evaluate the training program of the Project P.A.T.H.S. by using subjective outcome evaluation based on the perspective of the training participants. On the whole, the training program can be regarded as successfully implemented, and the training objectives were also well achieved. It also replicated the previous subjective outcome evaluation findings of the training program of the Experimental Implementation Phase –28. BaseThere are several strengths of this study. First, this study investigated different aspects of subjective outcome evaluation, including views of the participants towards the training program, training instructors, perceived effectiveness, and overall satisfaction, and all these scales were found to be reliable. Second, a respectable sample size was used in the study. Actually, it is noteworthy that there are very few scientific studies on the training program of positive youth development programs using such a large sample in the international context. Third, this study is one of the few empirical studies on the pre-implementation training program for the program implementers in a positive youth development program in Chinese context. Therefore, the present study provides a significant contribution to the literature.However, there are three limitations of this study. First, as only the first three levels of Kirkpatrick's model are discussed in this paper, the discussion is limited. In particular, the use of subjective outcome evaluation findings as support for Kirkpatrick's model should proceed with caution. Second, there is an alternative explanation for the present positive outcomes. Given the demand characteristic , the participants might give more positive evaluation or tend to focus on the positive side of the training. Nevertheless, this alternative explanation could be partially dismissed since all the participants were professionals and they were invited to reflect their views in a frank way with a serious manner before completing the questionnaire. Moreover, the questionnaires were anonymous in nature. Third, since the present findings are only based on quantitative data, collection of qualitative data with integration of the quantitative findings can give a comprehensive picture about the impact of the training programs. Despite these limitations, the present findings are pioneering addition to the Chinese database on positive youth development. Research studies have consistently showed that the Project P.A.T.H.S. was effective in promoting holistic development in Chinese adolescents –37. With


Young African American girls have a high risk of obesity. Online behavior change programs promoting healthy diet and physical activity are convenient and may be effective for reducing disparities related to obesity. This report presents the protocol guiding the design and evaluation of a culturally and developmental appropriate online obesity prevention program for young African American girls.The Butterfly Girls and the Quest for Founder’s Rock is an 8-episode online program delivered as an animated, interactive comic. The program promotes healthy diet and physical activity and is specifically designed for 8–10 year old African American girls. Girls, parents, and community representatives provided formative feedback on cultural relevance and developmental appropriateness. A three-group  randomized design (n = 390 parent/child dyads) is employed, with child as the unit of assignment. Change in body mass index is the primary outcome; change in fruit and vegetable consumption, water, and physical activity are secondary outcomes. Data collection occurs at baseline, approximately 3 months after baseline , and approximately three months later . Two dietary recalls are collected at each data collection period by trained interviewers using the Nutrient Data System for Research (NDSR 2012) system. Physical activity is objectively measured by seven days of accelerometry. Psychosocial and process data are also collected. Girls in the treatment and comparison groups will be interviewed at post 1 to obtain information on personal reactions to the program.This research will develop and evaluate the efficacy of an online program for reducing obesity risk among girls at risk of obesity and related diseases. Online programs offer the potential for wide dissemination, thus reducing disparities related to obesity.NCT01481948 Ethnic minorities in the United States have higher burden of chronic disease and death than non-minorities. BecauseInterventions must reflect the realities and expectations of the intended audience. ProgramLack of access to preventive programs has contributed to health disparities,20. UsinThe Butterfly Girls and The Quest for Founder’s Rock (BFG), an online obesity prevention program specifically designed for 8–10 year old African American girls.Although developmentally appropriate and culturally relevant online behavior change programs hold promise as an effective method for reaching youth-34, few This evaluation employs a three-group, randomized control design, with three data collection periods: baseline, post 1 (immediate post program completion), post 2 (3 months later). Following baseline assessment, girls are randomized to one of three groups . Girls in the treatment group engage in theory-prescribed behavior change procedures  embedded in online, interactive, role modeling comics. The comparison group receives the online comics without the theory-prescribed behavior change procedures. After participating in all three data collection periods , girls in the wait-list control group receive the treatment intervention. A wait-list group controls for potential threats to internal validity, such as history and maturation.Girls have up to three months to view all 8 episodes. For girls in the treatment and comparison conditions, post 1 data collection occurs immediately upon completion of the 8 episodes or approximately 3 months after playing episode 1, whichever occurs first; post 2 data collection occurs three months after post 1 , the volunteer database at the Children’s Nutrition Research Center (CNRC), and a video trailer posted to the CNRC volunteer website and newsletter. Prior to participation, written informed consent and assent is required. Child is the unit of randomization and assignment. Recruitment began in November 2012. The outcome study began enrolling participants in January 2013.The primary outcome is body mass index (BMI) percentile. Therefore, estimated sample size (SS) was based on the number of participants needed to detect a significant intervention group by time interaction effect for BMI percentile. Data from the Baylor GEMS study of 8–10 year old African American females yielded a baseline BMI percentile of 89.4 ± 14.7 n = 35)[[35]. GiParents and children participate in separate intervention and data collection activities electronically  from locations of their choice .Children view an eight-episode online program that promotes five servings of fruit and vegetables (FV), five glasses of water (40 fl oz), and 60 minutes of PA per day. Girls in the treatment group set and report goals, self monitor, and receive feedback related to level of goal attainment while comparison group girls do not engage in these activities.Fun, Food, & Fitness), guided by Social Cognitive Theory[The original online program was developed as part of the Girls Health Enrichment Multi-site (GEMS) Program initiative funded by the National Heart, Lung, and Blood Institute, National Institutes of Health, to identify ways to reduce obesity risk among 8–10 year old African American girls. The 12-e Theory and Elabe Theory,42, conse Theory,42. The Food, Fun, & Fitness Internet Program for Girls, a brief story introduction was added to provide context  (n = 78). High logon rates[The pilot test of the Baylor College of Medicine-GEMS project n = 35) demonstrated high camp attendance in the treatment group, but low login rates to the online program[ demonstron rates and staton rates. Media aon rates, suggestAlthough steps were taken to ensure cultural appropriateness and sensitivity in the original program, it was Three expert panels reviewed program components to ensure appeal and cultural and developmental appropriateness: girls (n = 20), parents (n = 20), and community representatives (n = 10). Girls and parents participated in interviews, and the community panel completed online surveys. Similar questions were asked of parents, girls, and community representatives. Interviews were conducted by trained staff following a semi-structured script; probes and prompts were used to expand and clarify responses. Interviews were digitally recorded and lasted approximately 1.5 hours each. Surveys were completed over a secure, password protected website. Stipends were provided for each interview  and survey ($50) completed.The first round of data collection reviewed the existing program to identify needed changes to structure and content. Modifications made as a result of the first round of data collection included changing the characters  and enhancing the behavioral components . A richer story was added, interactivity was enhanced, graphics were updated, sound effects were enriched, and a new vocal track was created. A second round of data collection was conducted to review the modifications. Although several minor modifications to characters’ appearances were suggested, the overall program met with the approval of parents, girls, and community representatives. The program is described below.The theoretical framework guiding the BFG program is comprThe Butterfly Girls and the Quest for Founder’s Rock.The original program did not have an overall storyline. Instead, online comics were vignettes loosely connected by setting and characters. To promote immersion and enhance appeal, a storyGirlfriends from MacGuffin Middle School are teased at camp by boys as they watch the girls take pictures of butterflies. As a result, the boys mockingly call the girls "butterflies." Back in school, the girls discover the boys have entered a citywide competition to find the legendary Founder’s Rock, which is the place where their town of MacGuffin Springs was first settled. The girls think the competition is a perfect opportunity to show the boys “who is superior to whom.” The girls decide to form The Butterfly Club and enter the competition. Each team has to run all over the city looking for clues. The first team to find Founder's Rock wins. Although the boys are sure the girls can't keep up, the girls discover that eating FV, drinking water, and being physically active help them gain the stamina needed to find the clues to win the contest. Despite seemingly insurmountable obstacles, The Butterfly Girls win the competition, eventually sharing their “secret” advantage  with the boys.The protagonists (non-player characters) are six young African American girls approximately 8–10 years of age Figure  and the BFG is a browser-based interactive comic developed in HTML5 using Hype 1.5 . The comics include voice-over narration performed by professional voice talent, a custom music score  and sound effects. Characters and settings were created as three-dimensional models and rendered as still images. The story genre is action adventure told from a second person perspective. Participants in the treatment group engage in activities designed to promote personal mastery , while those in the comparison group do not engage in these activities.The story unfolds in eight episodes. Art assets were created by a professional artist. The participant is represented in the story by an unseen character referred to as “Rookie,” since she is the newest member of the Butterfly Girls Club. Rookie has a dog named, "Wok" who makes humorous observations conveyed as thought bubbles. Character comments are voiced and also appear as on-screen text. Interactivity is included to help participants feel part of the story. For example, participants are encouraged to “explore” comic panel scenes to enhance immersion and enjoyment. At certain times, clicking on a character reveals their interior thoughts . Rolling over objects in a scene also elicits tooltips . A rich audio track with music and sound effects is included. The program structure is presented in Figure Multiple rounds of internal testing were conducted to identify technical issues, discrepancies between storyboards and the programmed episodes, and other issues that needed to be addressed prior to completing the program. This testing revealed that an Apple Safari or Google Chrome browser, Apple QuickTime, and Adobe Acrobat Reader  software were needed by participants. Links and directions regarding how to download and install these programs are available.Participants receive a unique username and password to logon to the online program. This information automatically routes them to their assigned version of the program  Figure .Girls in the treatment group set goals in each episode. To facilitate goal setting and attainment, girls are presented with schemas during goal setting. The schemas have been described elsewhere. SchemasCharacters and the study participant use a virtual "iPod" to set goals during the online program. Figure Girls in the treatment group are asked to keep track of progress towards their goals. The tracking form is presented in Figure This component appears in episodes 2–8 (treatment group only). Participants report their goal attainment via a “virtual” iPod. The schema they selected the previous episode appears on the computer screen, and girls report the days they met their goal for each behavior. Based on level of goal attainment, girls then receive feedback from one of the characters  Figure . ParentsA block randomization scheme is used to allocate subjects to the three groups . The block strategy ensures balance in the randomization process such that equal numbers of participants are assigned to each group. The computer algorithm written in SAS®  performed the block randomization using the PROC PLAN procedure and the computer system clock for the random seed.After completing baseline assessment, girls are randomized to condition. After randomization, girls in the treatment and comparison groups receive a “Club Member Package” delivered to them via courier service to welcome them into the study. The package includes a BFG logo t-shirt,a bracelet with “Butterfly Girls” stamped into it, and a 20 fl oz logo water bottle. A bottle of this size was selected because drinking two filled bottles a day will meet their water goal (five 8 fl oz servings). The package also includes a participant manual targeted to their study group. The participant manual for both treatment and comparison participants contains helpful information, such as the backstory , guidelines for navigating the website, and how to resolve technical problems. The manual for the participants randomized to the treatment condition also contains guidelines for how to set and report goals, as well as pre-printed tracking sheets. Girls randomized to the wait-listed control group receive these materials immediately prior to beginning the online program.Participants are eligible to play one episode each week after they complete the previous episode. This provides them with time to meet the goals they set the previous episode. They receive an automatic email each time an episode becomes available. Simultaneously, the parent receives an email with a link to an episode-specific newsletter. When a participant completes an episode, an email is automatically generated and sent to the research staff. The same procedure occurs when parents open a newsletter. If the participant does not view an episode within three days of eligibility to play the episode, they receive an email reminder. After six days, a reminder call is made to the parent. The parent then receives a call every five days until the child completes the episode . At this point, a decision is made as to whether to move the participant to inactive status. The program is structured so that girls will not miss an episode. For example, if they are eligible to play episode 2 but do not logon for three weeks, when they logon, they have access to episode 2.All self-report data are collected over a secure, password protected website. Parents and children are provided with separate passwords. A thorough process evaluation is also being conducted using the framework proposed by Baranowski and Jago. ExampleThe primary hypothesis is that girls randomized to the treatment group will maintain a stable BMI percentile, while girls in the comparison and wait list control group will show an increase. Secondary hypotheses will examine group differences in FV, water, PA, and psychosocial variables over time. A repeated measures analysis of covariance will be conducted to examine intervention effects at post 1 and post 2, covarying baseline values. All analyses will control for potential confounding variables .At three months post intervention, girls in the treatment group will maintain a stable BMI percentile, while girls in the comparison and wait-list control groups will have an increase. Additionally, girls in the treatment group will have higher FV and water consumption, more minutes of moderate to vigorous PA, and fewer minutes of sedentary activity than girls in the comparison or wait-list control groups immediately after the intervention and 3 months post intervention. It is also expected that home availability of FV and PA equipment will mediate intervention effects on BMI percentile, FV consumption, and moderate to vigorous PA. Based on results of a small pilot study (n = 16) conducted prior to the outcome evaluation, we also anticipate high participation rates for both the online program and data collection activities, low attrition, and high compliance with setting, reporting, and achieving goals.Online programs encouraging young African American girls to develop healthy diet and PA behaviors prior to adolescence are needed to reduce the risk of obesity and related diseases later in life. Procedures that contribute to behavioral change also need to be identified. Programs for ethnic minorities, however, must be culturally and developmentally appropriate and acceptable to key stakeholders, such as girls, parents, and the larger community to ensure community acceptance. They must also meet expectations of today’s savvy technology user in order to be adopted and used.The behavioral program described in this paper presents the protocol guiding the development of a culturally and developmentally appropriate obesity prevention program for 8–10 year old African American girls. This research will provide important insights regarding the efficacy of an animated, online program presented in a comic book format at changing behavior, and it will also contribute to our understanding of behavioral components that facilitate change in this age group.BMI: Body mass index; CNRC: Children’s Nutrition Research Center; F: Fruit; Fl oz: Fluid ounces; GEMS: Girls’ health Enrichment Multi-site Studies; PA: Physical activity; V: Vegetable.The authors declare that they have no competing interest.DT is principal investigator of the study, conceived this project; guided the design of the intervention components; and drafted the paper. RM assisted with development and took the lead in beta testing. RBhatt assisted with design and managed the project during development. CB and DC assisted with design. IV wrote the procedures for the dietary data collection. CC wrote the recruitment and enrollment procedures. KC was a co-investigator and assisted with design. TB was the principal investigator of the original study; as a co-investigator, he assisted with design of the current study. YL is the study biostatistician and assisted with sample size determination and analysis plan. CW created the storyline. RBuday was the executive producer for the online program and guided software development, creation of sound tracks, and programming. All authors read and approved the manuscript.RBhatt was with Department of Pediatrics, Baylor College of Medicine, when this program was developed; she is now with the Department of Family and Community Medicine, Baylor College of Medicine. CW is the playwright who wrote the story for the project. RBuday is the President of Archimage, Inc.  and served as the program's executive producer.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/13/709/prepub


Participation rates of lifestyle programs among type 2 diabetes mellitus (T2DM) patients are less than optimal around the globe. Whereas research shows notable delays in the development of the disease among lifestyle program participants. Very little is known about the relative importance of barriers for participation as well as the willingness of T2DM patients to pay for participation in such programs. The aim of this study was to identify the preferences of T2DM patients with regard to lifestyle programs and to calculate participants’ willingness to pay (WTP) as well as to estimate the potential participation rates of lifestyle programs.A Discrete Choice Experiment (DCE) questionnaire assessing five different lifestyle program attributes was distributed among 1250 Dutch adults aged 35–65 years with T2DM, 391 questionnaires (31%) were returned and included in the analysis. The relative importance of the program attributes  schedule, consultation structure, expected program outcome and out-of-pocket costs) was determined using panel-mixed logit models. Based on the retrieved attribute estimates, patients’ WTP and potential participation rates were determined.The out-of-pocket costs , consultation structure  and expected outcome  were the most important factors for respondents when deciding whether to participate in a lifestyle program. Respondents were willing to pay €128 per year for individual instead of group consultation and €97 per year for 10 kilograms anticipated weight loss. Potential participation rates for different lifestyle-program scenarios ranged between 48.5% and 62.4%.When deciding whether to participate in a lifestyle program, T2DM patients are mostly driven by low levels of out-of-pocket costs. Thereafter, they prefer individual consultation and high levels of anticipated outcomes with respect to weight loss. Participation rates of lifestyle programs among type 2 diabetes mellitus (T2DM) patients are less than optimal around the globe ,2. Yet, In order to obtain a better insight into the motives of T2DM patients for participating in lifestyle interventions, various studies were conducted that resulted in an extensive list of barriers for participation in lifestyle interventions as reported by T2DM patients –21]. Ho. Ho21]. A second omission in many studies on the willingness to participate in lifestyle programs is that the costs of lifestyle programs are often not included. If lifestyle interventions were to be implemented in ‘real life’, participants in such programs would have to pay at least part of the program costs out-of-pocket. Whereas participation in programs within a research setting tends to be free of charge. Taking into consideration that these programs need to be (partly) financed by the participants, it is worthwhile to examine the amount of money that potential participants would be willing to pay. Previous research among a non-diabetic population showed that the 582 individuals at a high risk of developing T2DM were willing to pay out-of-pocket for a lifestyle intervention . These rOnce we have obtained a better insight into and knowledge of program-related factors that are crucial for the participation of T2DM patients in these lifestyle programs as well as for their willingness to pay (WTP), recommendations can be made as to what type of program would most likely be preferred by its potential users. These recommendations can be taken into account when developing new lifestyle programs, thus increasing their reach, and hence their public health benefit.The aim of this study was to identify the preferences of diabetes type 2 patients for different characteristics of lifestyle programs. Based on these preferences, participants’ willingness to pay (WTP) as well as the potential participation rates of different lifestyle programs have been assessed.Within the Netherlands, generic diabetes care is arranged in care groups. A care group is a legal entity formed by multiple health care providers ) . A rando1c of 49.9 mmol/mol. Of all the respondents, 37.8% was overweight (indicated by a BMI between 25 and 30 kg/m2) and 41.1% was obese (indicated by a BMI higher than 30 kg/m2). The majority of the respondents reported no complications (78.3%) and no other chronic conditions (99.7%). 19% of the respondents reported that they did not use any form of medication, 67.2% reported that they used glucose lowering medication in the form of pills, 4.6% reported that they injected insulin, and 9.2% reported that they used both. Almost all respondents (94.1%) reported that their primary health care contact was their GP and not a specialist in secondary care . Self-management measures were applied by approximately half of this population; 45.8% reported that they monitored their HbA1c at home and 50.3% reported that they keep a T2DM diary.The mean age of the final study population was 57.2 years, more than half of the respondents were male (57.4%), almost all respondents were Dutch nationals 94.1%), and about half of the population (49.6%) completed higher secondary education or lower general professional education  software was used to develop a D-efficient design with 18 unique choice tasks ,31. To lThe draft questionnaire was pilot tested among a subgroup (n = 20) of the study population to ensure that the wording used in the questionnaire was correct and understood by the target population. Most of the pilot tests were performed by means of a postal questionnaire, respondents were requested to mark every question or answering category that they did not understand or found hard to grasp and they were asked to provide suggestions for improvement. Moreover, three think aloud pilot tests were conducted to obtain more insight into how the respondents approached answering the choice tasks. No changes in the attributes and/or levels were deemed necessary based on the results of this pilot study. Moreover, based on the pilot-test data, sample size calculations were performed to ensure that significant differences for each attribute could be detected at a 5% level ,32.NLogit 4.0  was used to construct the panel-mixed-logit (Panel-MIXL) models that were estimated within this study. When using such models, results are adjusted for the panel structure  of the data. As every respondent completed nine choice tasks, their answers may be correlated, which is then accounted for. Equation 1 was tested using these models.0 represents the alternative specific constant and β1 – β8 are the attribute estimates that indicate the relative importance of each attribute. Based on the model fit (AIC and Chi-square tests), the constant and the expected outcome attributes were set as random parameters .V describes the measurable utility of a specific lifestyle program based on the attributes that were included in the DCE. βeffect code 1 + βeffect code 2) [All nonlinear parameters were recorded into effect codes. In contrast to dummy coding, this coding procedure codes the reference category −1, so the sum of the effect coded attributes is always 0. The coefficient for the reference category is therefore −1*(β code 2) ,34.-v).In order to calculate patients’ marginal willingness to pay (WTP), the negative of the out-of-pocket attribute was used as a measure of the marginal utility of money. The ratio of either attribute estimate to this negative of the out-of-pocket attribute provides an estimation of patients’ WTP concerning that specific attribute ,35. MoreFor a more detailed description of the statistical methods used in this paper see Additional file Most of the attribute estimates were significant, indicating that they were important for T2DM patients when choosing whether to participate in a lifestyle program Table . ParticiSince the magnitude of the beta values depends highly on the coding of the attributes, attributes were recorded into the same coding scale  to enable the assessment of their relative importance. The results of the recoded analysis are not shown because they show a high degree of overlap with the results presented in Table The significant coefficient of the standard deviation of the expected outcome attribute indicates that there is indeed a high preference heterogeneity among respondents concerning the amount of weight loss they anticipate before starting the intervention.Finally, the opt-out option was chosen in 46.3% of the choice tasks and 23.5% of the respondents chose to opt-out in every choice task. Patient preferences did not change when these latter responders were excluded from the analysis.The WTP was calculated for the significant attributes only. The results show that respondents were willing to pay €21.0  for a switch from a flexible to a general PA schedule. Respondents were willing to pay an extra €127.8  per year for a lifestyle program organized via individual consultation instead of consultation with a group of 10 other patients. Respondents were willing to pay €96.8  per year for 10 kg anticipated weight loss.A lifestyle program that consists of all of the least preferred attribute levels  was set as the ‘base’ model. This program showed the lowest participation rate 48.5%) . Moreover, a lifestyle program with a general physical activity component, individual consultation and large expected outcomes in terms of weight loss was preferred. T2DM patients were willing to pay €21, €128 and €97 per year respectively for a lifestyle program with these desired levels of the attributes . Additionally, it was estimated that approximately 62% of the T2DM patients aged 35–65 years would participate in a lifestyle program with these preferred levels.Though there is limited evidence regarding T2DM patients’ preferences for a lifestyle program, the current findings are comparable to those of previously conducted DCEs among other target populations describing different lifestyle interventions. The study by Johnson  found thPatients in the current study reported to be willing to pay up to €97 per year for every 10 kilograms of anticipated weight loss. Though this seems a promising argument to boost participation without significantly increasing costs, one could argue that, despite the linearity of the initial costs and expected outcome attributes, beyond a certain point, participants are no longer willing to pay an additional €97 for an increase of 10 kilograms of anticipated weight loss. This was also shown by Johnson and colleagues who demonstrated nonlinearity of the WTP .Finally, results showed that a potential participation rate of 62% can be expected when attributes are operationalized in accordance with patient preferences. Current programs can be improved by organizing individual consultation and communicating clearly about the anticipated outcomes of the program . The participation rates found in this study show that T2DM patients are willing to participate if programs meet certain criteria. However, the participation rates do not exceed the most optimistic participation rates currently observed in lifestyle programs ,2,11. ThOur conclusions are restricted by a number of limitations. The usable response rate was 31.3%. As the current study was questionnaire-based and participation was on a voluntary basis, selective non-response seems plausible. For instance, the number of non-Dutch patients was relatively low. This may be due to language difficulties, as a good command of the Dutch language is needed to complete the questionnaire and especially the DCE tasks. Therefore, generalizability with regard to preferences of non-Dutch patient groups remains limited. Besides, it could be the case that patients who already perceive their lifestyle as being healthy chose not to participate in this study. However, in real life, it is not likely that these patients would participate in a lifestyle program and therefore these patients are of limited interest for this specific study.The current study included T2DM patients in the age group of 35–65 years, who do not suffer from severe diabetes-related complications. There is limited information on the representativeness of the current study population compared to the target population. Additional analyses of T2DM patients aged 35–65 in a large Dutch cohort study (EPIC-NL ) showed In conclusion, when deciding to participate in a lifestyle program, T2DM patients in the age group of 35–65 years are mostly driven by the out-of-pocket costs of a lifestyle program, the structure of the consultation and the expected outcome of the program. We therefore advise that lifestyle programs directed at T2DM patients should be set up based on individual consultation, while communicating about the expected outcomes of the program (in terms of weight loss) and keeping out-of-pocket costs as limited as possible.This research was funded by the Strategic Research fund of the National Institute of Public Health and the Environment. Sponsors were not part of the research group and therefore were not involved in conducting the research, writing the manuscript or submitting the final paper. None of the authors have stated that any competing interest exists.JV prepared the questionnaire, conducted the expert interview and focus groups, designed the DCE, performed the data analysis and wrote the manuscript. MSL contributed to all phases mentioned above. PFG and HAS were involved in writing the research protocol and have both critically reviewed the manuscript. JNS assisted in the recruitment of the care groups that distributed the questionnaire and critically reviewed the manuscript. GAW was the project leader of this research project and was therefore involved in all stages of this study. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/13/1099/prepubDetailed description of the additional questionnaire. Here a detailed description is provided about the content of the questionnaire that was distributed alongside the DCE questionnaire.Click here for fileDetailed description of the statistical methods. Here a detailed description is provided about statistical background end methods that were used for the analysis in this study.Click here for file


Cardiovascular risk management is largely provided in primary healthcare, but not all patients with established cardiovascular diseases receive preventive treatment as recommended. Accreditation of healthcare organizations has been introduced across the world with a range of aims, including the improvement of clinical processes and outcomes. The Dutch College of General Practitioners has launched a program for accreditation of primary care practices, which focuses on chronic illness care. This study aims to determine the effectiveness and efficiency of a practice accreditation program, focusing on patients with established cardiovascular diseases.We have planned a two-arm cluster randomized trial with a block design. Seventy primary care practices will be recruited from those who volunteer to participate in the practice accreditation program. Primary care practices will be the unit of randomization. A computer list of random numbers will be generated by an independent statistician. The intervention group (n = 35 practices) will be instructed to focus improvement on cardiovascular risk management. The control group will be instructed to focus improvement on other domains in the first year of the program. Baseline and follow-up measurements at 12 months after receiving the accreditation certificate are based on a standardized version of the audit in the practice accreditation program. Primary outcomes include controlled blood pressure, serum cholesterol, and prescription of recommended preventive medication. Secondary outcomes are 15 process indicators and two outcome indicators of cardiovascular risk management, self-reported achievement of improvement goals and perceived unintended consequences. The intention to treat analysis is statistically powered to detect a difference of 10% on primary outcomes. The economic evaluation aims to determine the efficiency of the program and investigates the relationship between costs, performance indicators, and accreditation.It is important to gain more information about the effectiveness and efficiency of the practice accreditation program to assess if participation is worthwhile regarding the quality of cardiovascular risk management. The results of this study will help to develop the practice accreditation program for primary care practices.This cluster randomized trial is registered at ClinicalTrials.gov nr NCT00791362 Cardiovascular diseases (CVD) remain an important cause of mortality and morbidity worldwide. In public health and in primary care, many efforts have been made to prevent CVD ,2. Althoe.g., a continuing education session or audit without follow-up). To enhance continuous improvement in primary care in the Netherlands, the Dutch College of General Practitioners (DCGP) initiated in 2005 a nationwide comprehensive practice accreditation program for primary care practices. This program consists of a systematic audit on the basis of validated performance indicators for diabetes mellitus, CVRM, asthma, chronic obstructive pulmonary disease (COPD), practice organization, patient experience, educational feedback to practices, the requirement to develop structured improvement plans, and a check on the implementation of these plans after one year. If the procedure is performed, primary care practices receive a certificate that provides accreditation for a time period of one year. While accreditation serves a range of purposes, improvement of professional performance and practice organization are prominent among these [The slow improvement of cardiovascular primary care may be caused by the one-off and condition-specific character of many improvement activities . The DCGP has developed a number of products and activities to implement these guidelines, including a national kick-off conference for general practitioners (GPs) in December 2005, and a supportive package  consisting of educational materials and software for assessment of cardiovascular risk. The practice accreditation program is an important approach to improve primary care, but controlled evaluations of its effect have not yet been done.A substantial number of performance indicators used in the practice accreditation program is related to CVD. These indicators are derived from the completely revised guideline on CVRM that was published by the DCGP late 2005 . The newThe overall aims of the study are to determine the effectiveness and efficiency of the practice accreditation program in primary care, focused on its effect on CVRM.Key objectives are:1. To determine the effectiveness of the program on primary performance indicators for CVRM by comparing practices in the accreditation program that focus their improvement plans on CVRM to practices in the accreditation program that focus their improvement plans on other domains of chronic care. Primary outcomes are documented controlled blood pressure, serum cholesterol, and prescription of recommended preventive medication .2. To determine the potential effect of the program on other indicators for CVRM, self-reported goal attainment in the intervention group, and unintended consequences. Secondary outcomes are all other indicators for CVRM, self-reported goal attainment in the intervention group, and unintended consequences .3. To determine the economic efficiency of the program in the observed period regarding the primary outcomes.4. To explore what factors and mechanisms are associated with change (or absence of change) of performance in CVRM.The study design is a cluster randomized controlled trial with a block design, considering primary care practices as units of clustering.The Medical Ethics committee Arnhem-Nijmegen waived approval for this trial. At follow-up practices will send a letter of invitation and informed consent to 100 patients with established CVD. Patients return their letter with informed consent to Radboud University Nijmegen Medical Centre with permission for audit of their medical records. The privacy of the participating patients will be protected, and all data will be coded and processed anonymously. For the baseline-measurement, mandatory information on indicators for patients with established CVD will be used, collected by practices on behalf of the practice accreditation program.Seventy primary care practices will be recruited from practices in the Netherlands who voluntarily apply to start the practice accreditation program. An invitation letter for the study will be sent as part of the instruction manual for the program. Practices with a clear preference for a specific improvement plan will be excluded from participation in the study while participating practices will be randomized to a group which starts with an improvement plan on CVRM or to a group that does not. This also accounts for practices that participate in regional programs for enhancing disease management.The study focuses on patients with established CVD. This includes patients with angina pectoris, acute myocardial infarction in their medical history, other chronic ischemic heart diseases, transient ischemic attack (TIA), ischemic stroke, peripheral arterial disease, and aneurysma aortae. Selection of patients out of electronic medical records with these conditions will be based on corresponding diagnostic International Classification of Primary Care codes , a worldwide system to label conditions in primary care .General practices will be the unit of randomization. A computer list of random numbers will be generated by an independent statistician and then used to randomly allocate practices to equally sized intervention group or control group. This will be done in a randomized block design in blocks of four practices in order of enrolment. We assume that improvement activities in the control group will not influence cardiovascular care.Visitatie Instrument Praktijkvoering, VIP) [The practice accreditation program is an existing procedure provided since 2005 by an independent body (NPA) that has a license to use the accreditation procedure developed by the DCGP. The DCGP remains responsible for the content and further development of the procedure; it will be responsible for adequate delivery of the practice accreditation program in this study. The practice accreditation program comprises, firstly, of a comprehensive audit  and written feedback to the practice, which covers a range of clinical domains (mainly chronic diseases), practice management, and patient experiences. The feedback, which consists of a comparison with benchmarks of other primary care practices, is discussed with a non-physician observer in a feedback consultation and helps to identify substandard performance. The second obligatory component, the planning of improvements in the practice according to the principles of quality management, is based on this feedback. The practice team is supported by a trained non-physician consultant. Practices that perform the procedure as planned are all accredited, so accreditation does not imply that a certain minimum score has been obtained . In the practice accreditation program, validated instruments are used: the Visit Instrument to asses Practice management  , clinicaThe intervention starts with volunteering for the practice accreditation program. After enrolment for the study, practices will be contacted by telephone for further explanation of the study protocol and to schedule the data collection. After data is collected, practices are randomized. Practices allocated to the intervention group are instructed to focus their improvement plans on cardiovascular diseases in the first year of the program.The control group also starts with volunteering for the practice accreditation program and follows the same routine as described for the intervention group. Practices allocated to the control group are instructed to focus their improvement plans on other domains than cardiovascular disease and diabetes mellitus (they may target CVD later after the study period of one year). They are instructed to focus their improvement plans on other clinical areas than CVD or diabetes.Both intervention and control group receive feedback on CVD indicators as part of the normal practice accreditation program. Practices in the intervention group are instructed to set targets related to process and outcomes of cardiovascular care . All practices will receive a minimum of four hours of support by outreach consultants for no cost, which is available in all regions. Also, the practices are provided with examples of improvement plans, which saves time and would make study participation more attractive.In each practice measurements are done at baseline and at follow-up Figure . At baseThe effect evaluation aims to determine the effectiveness of the program on primary performance indicators for CVRM and to determine the potential impact of the program on other indicators for CVRM, self-reported goal attainment in the intervention group, and unintended consequences.Primary outcomes have been selected from the 20 quality indicators for established CVD , which w1. The percentage of patients in the practice with known established CVD who have systolic blood pressure below 140 mmHg.2. The percentage of patients in the practice with known established CVD who have LDL cholesterol below 2.5 mmol/l.3. The percentage of patients in the practice with known established CVD with a record that aspirin, an alternative anti-platelet therapy, or an anticoagulant has been prescribed.Data concerning indicators are extracted from medical records and will be available at patient level so that linkage to other measures  can be made at patient level.2, measurement of waist circumference, fasting glucose measurement, influenza vaccination, registration of alcohol intake, control and advice for exercise and diet, and comprehensive risk assessment , BMI <25 kg/me.g., home care) and productivity losses, using a one-month retrospective observation period. Also, at follow-up in both groups, time and other resources of practice teams spent on quality improvement in the total period of 18 months will be documented. Data-collection on performance indicators will be done in the follow-up measurement by medical record audit.In follow-up measurements items of use of healthcare will be extracted from the medical records with a retrospective three-month observation period. These items include number of contacts in the practice , use of various types of cardiovascular medication, use of hospital care or other care providers for cardiovascular diagnosis or therapy. Additional information will be collected with patient questionnaires, particularly on other healthcare use . This will be measured in semi-structured telephone interviews.1. Exposure to other quality improvement activities: Both study groups report on their exposure to relevant professional education and practice improvement activities  will be added to measure health outcome . To mease.g., age, multi-morbidity, ethnicity at practice level) and practice characteristics [e.g., availability of nurses, delegation of medical tasks to assistance, practice size). Only factors emerging from previous research are considered to avoid overcorrection in the primary analysis. A logistic regression model will be constructed for each outcome to analyze these outcomes in relation to group  and measurement moment . Identified differences between the groups at baseline will be included in this analysis. Random coefficients will be included to allow for the clustering of data within practices. Each of the secondary outcomes  will also be analyzed in this way. Finally, if an internally consistent scale can be constructed (reflected by high reliability coefficients of the combined score), we will develop an aggregated measure of outcome, and use this in a similar random coefficients linear regression analysis.The study groups will be compared at baseline regarding known determinants of cardiovascular care and its improvement. These include patient factors ,28  and improvement related costs . Units of resources are monetary valued on the basis of prevailing Dutch guidelines  or natioThe process evaluation, the fourth key objective, aims to explore what factors and mechanisms are associated with change (or absence of change) of performance on CVRM indicators. Semi-structured telephone interviews will be held with primary care physician or the quality coordinator of the practice after data collection for follow-up measurement is concluded. Topics of this interview are: practice characteristics; feedback reports; composing improvement plans; and reasons to participate in and experiences with the practice accreditation program. We want to explore if specific elements of the practice accreditation program are the cause of change in CVRM. All interviews will be conducted by the same person and will be audio-recorded. Interviews will be transcribed verbatim. Two researchers will independently review the transcripts. Data analysis will be done according to the framework approach . Topics In the practices in the practice accreditation program up to 2006 (n = 139), the following median values at practice level were found on indicators referring to patients with CVD: 53% for acceptable blood pressure levels; 36% for acceptable cholesterol levels; and 38% for use of anticoagulents . These data suggest that the current scores on the primary outcomes are in the range of 35% to 55%, which imply that substantial improvement is possible in many practices. The proposed study has been powered to detect a difference of 10% (from 55% to 65%), not yet taking control for baseline values into account because of uncertainty regarding the correlation between baseline and follow-up measures.We expected that the accreditation and improvement program will have an effect of 5% to 10% absolute change, which is the median value of effect sizes in a comprehensive review of 235 studies on quality improvement . Other aThe study is planned from September 2008 until September 2012. In months 1 to 18, practices are recruited and included in the project and go through the accreditation procedure. The baseline data collection will take place in these months. During months 3 to 42, practices (in the intervention group) work on improving their management of CVD, practices in the control group on improvements in other areas. In months 18 to 42, follow-up measurements in intervention and control practices are planned. During months 43 to 48, data will be analyzed and reported.The sample of participating primary care practices in this study is composed of volunteers for the practice accreditation program and therefore not nationally representative for primary care practices in the country. This reflects current practice, in which practice accreditation is a voluntary activity. It implies that study results cannot be generalized to the  situation of obligatory accreditation. Furthermore, we only collect data on CVRM; therefore, we cannot make statements about the effects of the program on other chronic illnesses.e.g., involve a practice nurse). We intend to compare the groups with other, independent samples of practices that provide data on cardiovascular care to get an impression of the representativeness of our sample of practices.Because both intervention and control groups start with accreditation, this project cannot pick up non-specific effects of the practice accreditation program. For example, we expect that practices prepare for accreditation by improving their practice ; ICPC: International Classification of Primary Care; PACIC: Patient Assessment of Chronic Illness Care; TSRQ: Treatment Self-Regulation Questionnaire; ICER: Incremental cost effectiveness ratio; CEAC: Cost-effectiveness acceptability curve.MW is Co-Editor in Chief of Implementation Science; all decisions on this paper were made by another editor. EN, JL, EA declare that they have no competing interests. JB is involved in development of the practice accreditation program and receives funding for this from the Dutch College of General Practitioners. MB is an employee at the Dutch College of General Practitioners and is involved with the practice accreditation program.EN is involved in developing instruments, data-collection, analysis, and reporting aspects of the trial. MW, the project leader, is involved in all aspects of the study. JvL and JB are involved in the design of the study, the analysis, and the reporting. MB is involved in the design and reporting of the study. EA will evaluate the economics. All authors read and approved the final manuscript.