For many organizations, limited budgets and phased funding restrict the development of digital health tools. This problem is often exacerbated by the ever-increasing sophistication of technology and costs related to programming and maintenance. Traditional development methods tend to be costly and inflexible and not client centered. The purpose of this study is to analyze the use of Agile software development and outcomes of a three-phase mHealth program designed to help young adult Quebecers quit smoking.In Phase I, literature reviews, focus groups, interviews, and behavior change theory were used in the adaption and re-launch of an existing evidence-based mHealth platform. Based on analysis of user comments and utilization data from Phase I, the second phase expanded the service to allow participants to live text-chat with counselors. Phase II evaluation led to the third and current phase, in which algorithms were introduced to target pregnant smokers, substance users, students, full-time workers, those affected by mood disorders and chronic disease.Data collected throughout the three phases indicate that the incremental evolution of the intervention has led to increasing numbers of smokers being enrolled while making functional enhancements. In Phase I (240 days) 182 smokers registered with the service. 51% (n = 94) were male and 61.5% (n = 112) were between the ages of 18–24. In Phase II (300 days), 994 smokers registered with the service. 51% (n = 508) were male and 41% (n = 403) were between the ages of 18–24. At 174 days to date 873 smokers have registered in the third phase. 44% (n = 388) were male and 24% (n = 212) were between the ages of 18–24.Emerging technologies in behavioral science show potential, but do not have defined best practices for application development. In phased-based projects with limited funding, Agile appears to be a viable approach to building and expanding digital tools. Delivering interventions, improving patient safety, reducing cost, and accurately accessing and updating patient-centric information are the major driving forces behind the development of mHealth Despite the benefits and promise of mHealth there are significant costs in building and maintaining systems. Costs are generally divided into three types: direct (hardware), indirect (administrative and training) and intangible  For decades experts in technology and digital procurement have recognized the escalation of costs in software creation, maintenance, and reliability. As early as 1973 it was predicted that software maintenance and development costs would eventually outweigh the acquisition of hardware Traditionally, software development processes have been incremental, sequential, and inflexible. The most common example is the waterfall model, a sequential methodology where development flows downward. Waterfall, popular in large-scale enterprise projects, is largely falling out of favor as it does not allow for incremental change, flexibility, or rapid evolution The shift toward Agile software development framework is an attempt to solve the problems related to waterfall From cost projection or traditional project management perspectives, critics of Agile express concern that it is poorly defined, is not well published, and is regarded as more of a philosophy than a precisely defined methodology Approximately 17.3% of Canadians smoke, with Quebec having the highest percentage of smokers. In fact, smoking prevalence has increased in Quebec from 18.8% in 2010 to 19.8% in 2011 Only 3–5% of smokers are able to quit unaided However, there is very little published research into the feasibility of providing long-lasting, engaging digital mobile health  solutions on a population level http://www.smat.ca), an mHealth program designed to help young adult Quebecers quit smoking using Agile methodology. SMAT is maintained by the Société canadienne du cancer (SCC) Quebec Division.The purpose of this study is to describe phase-based development and evolution of Service de Messagerie Texte pour Arrêter le Tabac (SMAT: Anonymous retrospective data were extracted from SMAT's Structured Query Language (SQL) database. All data collection procedures adhered to international privacy guidelines As this study was based on unidentifiable archival data, it was deemed exempt from further review. Regulations for the collection, use, and disclosure of personal information set forth by the Office of the Privacy Commissioner of Canada, as outlined in the Personal Information Protection and Electronic Documents Act (PIPEDA), were adhered to. Data from the SMAT website are not publically available. However, researchers interested in reviewing datasets for academic purposes are encouraged to contact Société canadienne du cancer Quebec Division.The project was divided into three phases based on the availability of resources per fiscal year. A rough outline for the desired goal and timeline was developed with the funder. A phase-based approach was used to develop and expand SMAT. The original software code and each of the three phases of development followed Agile methodology and involved active input from multiple agencies including behavioral scientists, database architects, software developers, data analysts, marketers, funding agencies, suppliers, project managers and program managers see . The rathttp://www.SmokersHelpline.ca).The original source code was based on an experimental text-messaging program offered by Evolution Health Systems Inc (EHS). The program was based on the Health Belief Model edgy, alcohol, stress, slip, craving, smokers, motivate, stop).EHS and SHO adapted source code algorithms to appeal to two profiles: current smokers and recent quitters see . ReactivSample SHO Proactive Text Message:As your body adjusts to life without nicotine, edginess is common. Let others know you are not mad at them but just having a hard time.Emergency Coping Plan: AVOID or LEAVE the situation, DISTRACT, DELAY and use positive SELF TALK. Call SHL 1 877 513-5333 for support.SHO and EHS researchers collaborated on analyzing message frequency data, and presented findings at the international 2010 mHealth Summit in Washington DC In early 2010, Quebec-based SCC sought a novel solution that would engage and support young adults in their quit attempts. In order to make most efficient use of resources, SCC modified the SHO text-messaging platform to suit the unique needs and culture of Quebecers and the young adult target demographic. With a fixed budget and timeline, the project team agreed that the end result would be a text messaging service, but functionality would be determined through the course of development in agreement with Agile development principles. The goal for phase I was to enroll 100 users with 60% in the age range of 18–24.SCC and EHS engaged in a partnership to facilitate the tailoring process, while conforming to established behavior-change theories. First, a systematic literature review was conducted to establish best practices for delivery of text messaging in healthcare settings. Peer-reviewed literature, grey literature and white papers were included. Search themes includes smoking cessation interventions, young adults' smoking habits, young adults' motivations to quit, and existing text messages for smoking cessation.Next, tone language and content were tested in four focus groups, two in Montreal and two in Quebec City. Criteria of attendees were age , a desire to quit smoking in the next six months, the use text messaging on a daily basis, and current smoking. Eight participants were randomly selected for each of the four focus groups.help, commitment and coach should be avoided. Elements of brief intervention Findings indicated that young adults desired tips and information on physiological changes due to cessation, and that a mature tone should be utilized within messaging. Other findings indicated that the free nature of the program should be promoted, that the program was offered by a provincially recognized not-for-profit was viewed positively, and that the words After EHS and SHO collaborated on the design of the new program, a one-week test simulating the service was administered to 12 young adults who were randomly recruited from the general population and wanted to quit smoking. Following the test, 20-minute phone interviews were conducted for feedback. Messages were created in an iterative process, and were based on results from the literature review, focus group testing, simulation testing, and user feedback see .After program modification and quality assurance testing Phase I was launched in August of 2010. The pilot program consisted of proactive messages that were strategically delivered according to the user's quit date see .distraction, ennui, pause, oups, party, stress). A library was created to recognize common misspellings and to ask the user to clarify if no alternatives were found.In addition to modification of proactive messages, six reactive keywords were implemented into the program . Of particular interest in the review of data were conversations that individuals appeared to have with the reactive keyword service see .Similarly, qualitative feedback from 54 participants indicated that 40% (n = 22) would like to live text chat with a quit specialist. Based on the recurrence of individuals initiating conversations with the reactive service, a text-chat service named Text and Chat Integrated (TaChI) was created.The TaChI web interface allowed telephone quit counselors, from La Ligne j'Arrête, a service also provided by la Société canadienne du cancer who were already part of the existing infrastructure, the ability to engage in real-time text-chats with registered members see . Two speUsing Agile methodology, the TaChI interface was co-created by an interdisciplinary team. Informal qualitative interviews and testing processes were held with telephone quit specialists (n = 8) to assess need and functionality, and design interfaces were collaboratively developed with database architects (n = 2) and program managers (n = 3).Phase II was launched December 6, 2011. Throughout Phase II functional upgrades were made to the TaChI interface based on telephone quit specialist needs, and additional datasets were coded for future analysis. The goal for phase II was to enroll 1,000 users with 60% in the age range of 18–24. Phase II came to a conclusion on March 1, 2012 with text survey data being collected until September 30, 2012.Phase III was launched March 15, 2013. Data analysis, usage patterns, and input from users and quit specialists were used to enhance SMAT algorithms. Using Agile as a strategy, major upgrades included the ability for users to change quit dates, stop and restart the service, and development of a relapse-recovery algorithm.To improve personalization, registrants had the option to select tailored algorithms for one or more of six specific conditions: pregnant smokers, those who use other substances such as cannabis, alcohol, or coffee, quitters with mood disorders, students, full-time workers, and those with a chronic disease. Based on their selections the system algorithm delivered tailored messages.ennui and difficile) and one keyword was removed due to its limited use (voyage). The program was extended from 12 weeks to 24.Two additional reactive keywords were added to the service  of registrants in Phase II and 60% (n = 523) in Phase III. Use of TaChI has also intensified; in Phase II (300 days) there were 1,099 messages exchanged, compared to 3,262 so far in Phase III (174 days). The use of TaChI, and the fact that registrants can receive instant feedback when they need it most, reflects past research indicating immediate responses from digital health tools assist in relapse prevention A further indication of instant support is the fairly consistent use of reactive keywords by registrants . This may also be an indication of SMAT being utilized for relapse-prevention purposes.Preliminary Phase III results indicate many participants fall into at least one of the six tailored profiles. For example, 44% (n = 385) voluntarily registered for the full-time worker program, and 36% (n = 318) voluntarily registered to receive messages about quitting and substance use. While only 1% of registrants (n = 11) were included in the pregnant smokers algorithm, the percentage reflects the population pregnancy frequency, and this may be encouraging as pregnant smokers are often stigmatized.In Phase II, 8% (n = 83) reset their quit date. There was an increase in the use of this program enhancement in Phase III, with 13% (n = 115) of registrants using this tool.Similar to resetting a quit date 6% (n = 55) of users in Phase II restarted the service from where they had left off (if they had stopped the program). Half as many users, 3% (n = 28), have used this feature to date in Phase III.Considering high rates of smoking among 18–24-year-olds in Quebec, it is encouraging that SMAT is attracting users within this age group. However, usage has been somewhat inconsistent with 62% (n = 112) of registrants being between the ages of 18–24 in the pilot phase, 41% (n = 403) in Phase II, and 24% (n = 212) to date in Phase III see .The use of the program by both males and females is also encouraging, with 51% (n = 94) of registrants being male in the pilot phase, 51% (n = 508) in Phase II, and 44% (n = 388) in Phase III.Also fairly consistent is the average number of cigarettes per day. The pilot phase and Phase II attracted users who smoked an average number of 17 cigarettes per day, while Phase III registrants smoked 20 cigarettes per day.At the conclusion of Phase I and II, follow-up surveys were conducted via telephone interviews, web and text surveys.Following phase I, 28% (n = 74) of registrants were surveyed by web and telephone. 91% (n = 67) reported that they would “definitely” or “probably” recommend SMAT to a friend who was quitting smoking, 86% (n = 64) were satisfied with the service, 73% (n = 54) believed SMAT was useful in their quit attempt, and 57% (n = 42) would have liked SMAT to continue for longer than 12 weeks.Of survey participants, 43% (n = 32) indicated they had not smoked in the past seven days, resulting in an intent-to-treat quit rate of 12%.Following Phase II, results from qualitative in-depth telephone interviews (n = 249) and web surveys (n = 124) measured satisfaction with the service. Results indicated that 71% (n = 265) wanted to continue to receive text messages, 62% (n = 231) said SMAT was useful in their quit attempts, and 95% (n = 354) would recommend the SMAT service to others.Also following Phase II, 3-month follow-up surveys were conducted via text messaging. A total of 27% (n = 273) of registrants responded with 29% (n = 76) indicating abstinence since their quit date. Although not a randomized trial the 3-month self-report intent-to-treat quit rate was 7.6%. It is important to note that results from interviews and surveys are observational.The goal of this study was to assess whether Agile software development could efficiently build and expand an mHealth intervention through separate phases of funding. Results indicate that despite limited budgets and time restrictions, incremental development through Agile expanded the functionality and value of the program.Of particular interest is the functional evolution TaChI from Phase II to Phase III. The creation of TaChI was not envisioned at project onset and its development is a result of Agile. Had a traditional methodology to the program development been employed such as waterfall, where project phases and scope are fixed early on, it is unlikely that the TaChI would have been developed.The modification of reactive messages based on user need increased service usage. At program onset, there were six reactive keywords. In Phase III 12 keywords are now offered. Qualitative analysis of keyword use, misspellings, and common subjects in text chats can improve the functionality and scope of this popular component.The Phase III development of algorithms targeting six specific conditions is also popular. Agile can also be applied to the development of this type of software component, with the intent of having targeted messages reach special populations with specific needs.While working within limited budgets and timeframes, Agile can be used in the strategic development of a highly utilized mHealth program that is used by young Quebecers, a group that traditionally fails to seek treatment.In the Phase I, use amongst young adults aged 18–24 was 62% (n = 112). This decreased to 41% (n = 403) in Phase II, and then decreased 24% (n = 212) in Phase III. While it may be explained through external factors such as seasonality or changes in promotional strategies, age variance should be investigated following Phase III.From a funding perspective, the ongoing iterative process involved with Agile may be identified as a business risk If Agile methodology is continued to be followed beyond Phase III, the project will continue to seek input from behavioral scientists, statisticians, program managers, database designers, project managers and quit specialists. This use of interdisciplinary experts can continue to help assess which SMAT elements are most relevant to the project's objectives, and which elements should be enhanced, added or removed.In summary, the present study suggests that Agile is a feasible strategy that can transform small-scale digital health projects into full-scale programs with enhanced functionality. Additional studies may help smaller organizations understand how to maximize budgets though interdisciplinary methodologies like Agile.Emerging digital health technologies are promising, but do not have defined best-practice application to behavioral science. Due to its iterative nature and relationship to experimental design, Agile is a viable approach for cross-functional teams.Adopting Agile requires modifications to traditional approaches. As a methodology that lacks an incremental, sequential process it may not be possible to utilize traditional project management software to illustrate predictive schedules. Program improvements are iterative and until they have been fully tested in a user population, they can be subject to errors. To overcome these issues a high degree of communication and cooperation between team members is required.The primary challenge the SMAT team overcame was to utilize Agile within fixed and limited budgets, and distinct phase timelines. Technical enhancements and future phases will become increasingly complex. For example, the next incremental phase could involve transforming SMAT into both an SMS and mobile application (app). An app would allow for integration with social networking, which could further support relapse prevention. However, a significant cost challenge would be adapting the source code for compatibility with multiple mobile operating systems.Further work is required in healthcare outcomes economic research that establishes development and maintenance strategies for software designed to have an ongoing population effect.


Based on the consolidated data with schools as units, results showed that the participants generally had positive perceptions of the program, implementers, and benefits of the program. More than four-fifths of the participants regarded the program as beneficial to their holistic development. Multiple regression analysis revealed that the perceived qualities of the program and the program implementers predicted perceived effectiveness of the program. Based on the subjective outcome evaluation findings, the present study provides support for the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. in Hong Kong.Subjective outcome evaluation findings based on the perspective of the participants of the Project P.A.T.H.S.  in nine datasets collected from 2005 to 2009 ( In his review of adolescent developmental issues in Hong Kong, Shek  drew sevTo promote holistic development among adolescents in Hong Kong, The Hong Kong Jockey Club Charities Trust approved HK$400 million to launch a project entitled “P.A.T.H.S. to Adulthood: A Jockey Club Youth Enhancement Scheme” based on the perspective of positive youth development. The word “P.A.T.H.S.” denotes Positive Adolescent Training through Holistic Social Programmes. There are two tiers of programs (Tier 1 and Tier 2 Programs) in the P.A.T.H.S. Project. Whereas the Tier 2 Program is a selective program for students with greater psychosocial needs, the Tier 1 Program is a universal positive youth development program in which students in secondary 1 to 3 participate, normally, with 20 h of training in the school year at each grade, involving 40 teaching units that have been developed with reference to 15 positive youth development constructs [There were two implementation phases in the original phase of the project—the experimental implementation phase and the full implementation phase. For the experimental implementation phase (January 2006 to August 2008), 52 secondary schools participated in the project with the objectives of accumulating experience in program implementation and familiarizing the front-line workers with the program design and philosophy. In the 2006/2007 school year, the programs were implemented on a full scale at the secondary 1 level. In the 2007/2008 school year, the programs were implemented at the secondary 1 and 2 levels. In the 2008/2009 school year, the programs were implemented at the secondary 1, 2, and 3 levels. The experimental and full implementation phases for the first cycle were successfully completed [To provide a comprehensive picture pertaining to the effectiveness of the project, a wide range of evaluation strategies were employed to examine the program effect, including objective outcome evaluation utilizing a randomized group trial; subjective outcome evaluation based on quantitative and qualitative data collected from the program participants and instructors; qualitative evaluation based on focus groups involving students and instructors; in-depth interviews with program implementers; student products, such as weekly diaries; process evaluation involving systematic observations of delivery of the program and interim evaluation. The available evaluation findings consistently provide strong evidence that the Project P.A.T.H.S. has a beneficial influence on students –9.To examine the perceptions of the program participants concerning the effectiveness of the project, subjective outcome evaluation or the client satisfaction approach was used. In human services, the importance of involving service users or program participants in evaluation is advocated, and thus subjective outcome evaluation becomes popularly used to capture the viewpoints of the participants. To capture the viewpoint of the participants, client satisfaction surveys are commonly used as feedback for transforming services, to meet the users' needs for planning and administration purposes, or simply as an indicator of program effectiveness from the participants' perspective for research purposes. Although there are many criticisms of this approach, the client satisfaction approach is widely used in different service settings. As pointed out by Royse , “despitSubjective outcome evaluation is a popular approach employed by different professionals in different fields, such as education, social work, psychology, medicine, and allied health professions. The commonly used method develops closed-ended rating scale items to quantify client satisfaction. For example, standardized rating scales, such as the medical interview satisfaction scale, consumer satisfaction questionnaire, and client satisfaction questionnaire, were developed to gauge client satisfaction and perceived helpfulness of the program. In fact, it is commonly argued that, with the use of valid and reliable measures of the perceptions of the program participants, subjective outcome evaluation can yield objective pictures about program evaluation.Previous studies showed that roughly four-fifths of the program participants generally had positive perceptions of the program, instructors, and benefits of the P.A.T.H.S. Project. In addition, the findings are fairly stable in different cohorts of students in the experimental and full implementation phases –13. FurtFrom 2005 to 2009, the total number of schools that participated in the Project P.A.T.H.S. was 244, with 669 schools in the secondary 1 level, 443 in the secondary 2 level, and 215 in the secondary 3 level . AltogetAfter completing the Tier 1 Program, the students were invited to respond to a Subjective Outcome Evaluation Form for Students (Form A) developed by the first author. From 2005 to 2009, a total of 206,313 questionnaires were completed . The overall response rate was 92.48%. To facilitate the program evaluation, the research team developed an evaluation manual with standardized instructions for collecting the subjective outcome evaluation data. In addition, adequate training was provided to the implementers during the 20 h training workshops on how to collect and analyze the data collected by Form A.On the day when the evaluation data were collected, the purpose of the evaluation was mentioned and the confidentiality of the data collected was repeatedly emphasized to all of the respondents. The respondents were asked to indicate if they did not want to respond to the evaluation questionnaire . All respondents responded to all scales in the evaluation form in a self-administration format. Adequate time was provided for the respondents to complete the questionnaire.The Subjective Outcome Evaluation Form (Form A) –13 was uparticipants' perceptions of the program, such as program objectives, design, classroom atmosphere, interaction among the students, and the respondents' participation during class (10 items)participants' perceptions of the workers, such as the preparation of the instructor, professional attitude, involvement, and interaction with the students (10 items)participants' perceptions of the effectiveness of the program, such as promotion of different psychosocial competencies, resilience, and overall personal development (16 items)the extent to which the participants would recommend the program to other people with similar needs (1 item)the extent to which the participants would join similar programs in the future (1 item)overall satisfaction with the program (1 item)things that the participants learned from the program (open-ended question)things that the participants appreciated most (open-ended question)opinion about the instructor(s) (open-ended question)areas that require improvement (open-ended question).For the quantitative data, the implementers collecting the data were requested to input the data into an EXCEL file developed by the research team that would automatically compute the frequencies and percentages associated with the different ratings for an item. When the schools submitted the reports, they were also requested to submit the soft copy of the consolidated datasheets. After receiving the consolidated data by the funding body, the data were aggregated in order to “reconstruct” the overall profile based on the subjective outcome evaluation data by the research team.Percentage findings were examined using descriptive statistics. A composite measure of each domain  was created based on the total scores of each domain divided by the number of items. Pearson's correlation analysis was used to examine if the program content and program implementers were related to the program effectiveness. Multiple regression analysis was performed to compare which factor would predict the program effectiveness. All analyses were performed by using the Statistical Package for Social Sciences version 17.0.The quantitative findings based on the closed-ended questions are presented in this paper. Several observations can be highlighted from the findings. First, the participants generally had positive perceptions of the program , includiα = 0.98), 10 items related to the implementer (α = 0.99), 16 items related to the benefits (α = 1.00), and 36 items measuring overall program effectiveness (α = 0.99). Results of correlation analyses showed that both program content  and program implementers  were strongly associated with program effectiveness . Further analyses showed that program content (β = 0.75) was a significantly stronger predictor than program implementers (β = 0.24). This model explained 95% of the variance toward the prediction of program effectiveness. Interestingly, the above relationships and the amount of variance were consistent across grade levels.n = 206, 313) were involved. Second, different datasets collected at different points of time were analyzed in this study. Third, responses of students in different grades were collected. Fourth, this is the first known scientific study of the subjective outcome evaluation of a positive youth development program based on different cohorts in China. Finally, this is also the first study of subjective outcome evaluation based on such a large sample of participants in the global context.The purpose of this study was to evaluate the Tier 1 Program of the Project P.A.T.H.S. via the subjective outcome evaluation approach based on the perspective of the program participants using the data collected in the experimental and full implementation phases (2005–2009) of the project. There are several characteristics of this study. First, a large sample of schools (more than 200 schools per grade) and students (Generally speaking, the quantitative findings showed that a high proportion of the respondents had positive perceptions of the program and the workers; roughly four-fifths of the respondents regarded the program as helpful to them. The findings basically replicated those findings reported previously based on the perspective of the program participants and they are also consistent with those based on the perspective of the program implementers. In fact, an examination of the percentages of responses to different items revealed that the figures were very similar across different studies. In conjunction with findings based on other evaluation strategies, the present integrative evaluation study showed that the Tier 1 Program of the Project P.A.T.H.S. was well received by the program participants, and over four-fifths of them were of the view that the program was beneficial to their development.There are several contributions of the present study. First, in view of the lack of positive youth development programs and related evaluation findings in the Chinese context, the present study is a pioneer study. Besides showing that Project P.A.T.H.S. is effective, it also demonstrates how subjective outcome evaluation based on a large sample size can be carried out. Second, the findings show that the subjective outcome measure is reliable. Because there are few validated measures in the Chinese culture , 15, theFinally, findings on the predictors of subjective outcome evaluation are important because there are currently few conceptual models on the determinants of subjective outcomes. There has been some discussion in the literature on how the quality of a program can be enhanced by tailoring an appropriate program to suit the values and needs of target populations , 17. ForAlthough utilization of subjective outcome evaluation or the client satisfaction approach in evaluation has a long history in human services, there are arguments against the use of subjective outcome evaluation. For example, subjective outcome evaluation has been criticized as biased and unable to reflect the real behavioral changes in the program participants , 20. NevThird, because the findings reported in this paper were “reconstructed” based on the reports submitted anonymously by the participating schools, the possibility that the students reported in an over-cooperative manner was not high. Finally, previous research findings based on the same project have shown that subjective outcome evaluation findings actually converged with objective outcome evaluation findings , 22. In Despite these limitations, the present findings suggest that the Tier 1 Program of the Project P.A.T.H.S. and its implementation were perceived in a positive manner by the program participants. In conjunction with other evaluation findings, the present study suggests that the Tier 1 Program of the Project P.A.T.H.S. was perceived to be beneficial to the development of the program participants. With reference to the gradual decline of parental control in the early teenage years of Chinese adolescents in Hong Kong, positive youth development programs such as the Project P.A.T.H.S. are important initiatives to promote psychosocial competencies in Chinese adolescents of Hong Kong . Further


The authors would like to add the following funding sources to their funding information: "This study is also supported by National Key Scientific Project of China (973 Program #2011CB933500) and the Priority Academic Program Development of Jiangsu Higher Education Institutions，PAPD."


PAIRS Program and its effectiveness in enhancing medical education as a service-learning activity and replication model for the Buddy ProgramTM.As life expectancy increases, dementia incidence will also increase, creating a greater need for physicians well-trained to provide integrated geriatric care. However, research suggests medical students have limited knowledge or interest in pursuing geriatric or dementia care. The purpose of this study is to evaluate the Between 2007 and 2011, four consecutive classes of first year Boston University School of Medicine students  participated in a year-long program in which they were paired with a patient with early-stage Alzheimer’s disease (AD). Assessments included pre- and post-program dementia knowledge tests and a post-program reflective essay.t-test revealed a modest improvement in dementia knowledge post-program (p < 0.001). Using qualitative coding methods, 12 overarching themes emerged from the students’ reflective essays, such as observing care partner burden, reporting a human side to AD, reporting experiences from the program that will impact future clinical practice, and obtaining a greater understanding of AD.Program completion was 100% (n = 45). A paired-sample PAIRS Program can enhance the acquisition of knowledge, skills, and positive attitudes regarding geriatric healthcare in future generations of physicians, a skill set that is becoming increasingly relevant in light of the rapidly aging population. Furthermore, results suggest that The Buddy ProgramTM model can be successfully replicated.Quantitative and qualitative findings suggest that the  Alzheimer’s disease (AD) affects an estimated 5.3 million older Americans. While tPartnering in Alzheimer’s Instruction Research Study (PAIRS) Program is one unique service-learning program aimed at enhancing geriatric healthcare issues in medical education. The program’s student-focused objectives include: (1) educating students about AD and related cognitive impairment; (2) familiarizing students with care and support-related issues patients and their families encounter; (3) improving students’ communication skills with elders and cognitively impaired individuals; and (4) introducing students to career opportunities in geriatrics and related fields.The Boston University PAIRS Program is the first educational initiative to replicate the Northwestern University Buddy ProgramTM. The current study aims to quantitatively and qualitatively evaluate the PAIRS Program in fulfilling the four learning objectives outlined above. We hypothesized that dementia knowledge would quantitatively increase post-program and that qualitative analysis of end-of-the-year reflective essays would yield themes aligned with the PAIRS Program objectives.The PAIRS Program, which included a written application and interview process. In September, students were selected for participation based on enthusiasm, professionalism, and commitment level for the program requirements across the academic year. Selected students completed pre-program dementia knowledge tests upon program enrollment . Students then participated in three hours of formal lectures on AD and dementia fundamentals and communication skills for interacting with aging and cognitively impaired adults. Students were paired with an early-stage AD “buddy”  and introduced to their “buddy” at a Match Day event in October. Between November and May, the “pairs” met monthly for a minimum of four hours and participated in social and cultural activities, such as dinner or visiting a museum. Students reported reactions via activity journal entries and attended monthly luncheons to receive supplemental education from guest lecturers  and share experiences with each other and program staff. Students completed post-program dementia knowledge tests following the final program meeting in May. Students received elective credit upon completion. The Institutional Review Board approved the use of program data for research and determined the research was exempt from written consent in accordance with federal regulation 45 CFR 46.101(1). While written consent was not obtained, all students provided assent to allow their de-identified program data to be used for research purposes.Starting in 2007 and repeating annually, first year Boston University medical students were invited to apply to the A 33-item measure assessed dementia knowledge pre- and post-program. Scores In 2009, a 64-item measure was developed in an effort to more comprehensively assess dementia knowledge pre- and post-program. This measure was first implemented at the beginning of program year 3 and therefore completed by a subset of students enrolled in program years 3 and 4. Scores range 0–64 with higher scores indicating increased knowledge.PAIRS Program experiences, including what they knew about AD before beginning the program, what they learned during the program, and how their participation will affect their medical career.Students write an end-of-the-year essay reflecting on their Descriptives were generated for the student applicants and participants. Pre- and post-program knowledge scores were compared using paired-sample t-tests with a post-hoc item analysis. Analyses were conducted using SPSS 16.0 . Significance was set at p < 0.01 for primary analyses and p < 0.001 for post-hoc comparisons.To qualitatively analyze the essays, Consensual Qualitative Research strategies were employed. Two sta(62) = −1.32, p = 0.19), sex , education (t(77) = −0.47, p = 0.64), prior professional experience with an individual with AD , or personal/familial familiarity with AD . See TableBetween 2007 and 2011, 4 program years were completed with 79 student applicants and 45 enrollees out of 700 total medical students enrolled between 2007 and 2010. Comparison of admitted (n = 45) versus not admitted applicants (n = 34) revealed no between-group differences for age (t(43) = 0.47, p = 0.64) or personal familiarity with AD (t(43) = 0.09, p = 0.93).Dementia knowledge test performance at program entry as assessed by the Buddy Program knowledge test was unrelated to students’ prior professional experience with AD (t(44) = −6.3, p < 0.001). Inspection of mean pre- and post-program performances suggests an improvement of 2.5 items, corresponding to 7.5% enhanced accuracy  = −5.1, p < 0.001). Inspection of mean pre- and post-program performances suggests an improvement of approximately 4.5 items, corresponding to 7% enhanced accuracy  who completed the Boston University PAIRS Program Dementia Knowledge Test  and the purposeful selection of our students, generalizability of our findings to a larger medical student population may be limited. Because only students interested in the program goals elected to apply and the students selected were among the more enthusiastic and committed students who interviewed, it is possible that our participants’ engagement in the program does not reflect that of the average first year medical student. It is noteworthy that comparison of our participants with applicants not selected for participation suggests comparable pre-existing exposure to and personal/familial familiarity with AD. However, it is still plausible that all applicants, whether selected or not selected for participation, are fundamentally different from their classmates who did not apply for the program. Another noteworthy limitation is that while there was a modest improvement on the pre-and post-tests of Alzheimer’s disease and dementia knowledge, the source of improvement is unknown. That is, it is unknown if enhanced knowledge over the academic year was due to the monthly student luncheons with supplemental education, monthly buddy interactions, or some combination of these curricular elements. Themes from the student’s reflective essays support the notion that a combination of the monthly luncheons and the buddy interactions enhanced student knowledge; however, future evaluation methods should attempt to better understand the most valuable element of the program in enhancing student knowledge.PAIRS Program and expansion of existing evaluation methods. We will continue to track the residency selections for program alumnae with each class of graduating students. In program year 5, we introduced quantitative assessment of changes in students’ attitudes, stigma, and empathy via survey tools pre- and post-program participation. These measures are designed to more comprehensively monitor the development of key elements of humanism in medicine. Research is also needed to evaluate prospective replication of the program model among other student cohorts, such as nurses, social workers, and undergraduate students. Finally, evaluation methods could be enhanced by addition of buddy and care partner feedback.Future directions include continued assessment of the PAIRS Program and its effectiveness in enhancing medical education as a service-learning. Quantitative and qualitative analyses of data from the first four program years show post-program improvements aligned with the program objectives. Performance on knowledge tests and themes extracted from students’ reflective essays demonstrate an increase in dementia knowledge, appreciation for the psychosocial and support-related challenges faced by patients and families, enhanced communication skills, and exposure to dementia-related fields of medicine and research following completion of the program. Our findings suggest the PAIRS Program can successfully fulfill the need for service learning opportunities while improving positive attitudes regarding the healthcare of older or cognitively impaired adults. Increased interest in geriatric healthcare is more important than ever in light of the growing needs of our rapidly aging population. Moreover, our study supports the ability to successfully replicate The Buddy ProgramTM model and encourages additional institutions to implement similar programs to enrich students’ medical education.This study evaluated the The authors declare they have no competing interests.AJ and DM designed the study. NC and LB conducted qualitative analyses, and AJ conducted statistical analyses. AJ, NC, LB assisted with interpretation of results. AJ, NC, LB, and DM assisted with critical revision of the manuscript. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/12/80/prepub


From 2006 to 2009, the Dutch government provided €5 m annually for a nationwide program to reduce seclusion in psychiatric hospitals by 10% a year. We aimed to establish whether the numbers of both seclusion and involuntary medication changed significantly after the start of this national program.Using Poisson regression to estimate difference in logit slopes, we analyzed data for 1998–2009 from the Dutch Health Care Inspectorate, retrospectively examining the national numbers of seclusion and involuntary medication before and after the start of the program.The difference in slopes of the numbers of seclusion before and after the start of the program was statistically significant (difference 5.2%: p < 0.001). After the start of the program seclusions dropped 2.0% per year. Corrected for the increasing number of involuntary hospitalizations this figure was 4.7% per year. The difference in slopes of the numbers of involuntary medication did not change statistically significant . After correction for the increasing number of involuntary hospitalizations the difference turned significant .After the start of the nationwide program the number of seclusions fell, and although significantly changing, the reduction was modest and failed to meet the objective of a 10% annual decrease. The number of involuntary medications did not change; instead, after correction for the number of involuntary hospitalizations, it increased. The coercive intervention of first choice in most European countries is involuntary medication . In the This retrospective study was intended to establish whether the number of seclusions had changed significantly since the start of this program. As the program focused mainly on reducing seclusion and not on other coercive interventions, our second objective was to determine whether there had been a concomitant increase in the use of involuntary medication.In 2006, grants were awarded to 34 Dutch psychiatric hospitals , a number that had increased to 42 by 2009 (approximately 90%). The grants were allocated only to psychiatric hospitals that had a specific plan how to reduce the number of seclusions , with program coded as 0 before the start of the program in 2006 and 1 afterwards. Slope1 is the exponential slope before 2006. Slope2 is the difference between slope1 and the exponential slope since 2006. To aid interpretation, the slopes are presented in annual percentages of increase or decrease. Analyses were performed on quarterly data.A Poisson regression in the R statistical package , they fell significantly between 2006 and 2009 to an annual decrease of 2.0% .The use of involuntary medication had increased by 8.5% annually from 1998 to 2005 . Between 2006 and 2009, this increase was 8.0%  see Figure per involuntary hospitalization from 1998 to 2005 (logit slope = 0.967) was followed between 2006 and 2009 by a significantly greater annual decrease of 4.7% .A 3.3% annual decrease in the number of seclusions per involuntary hospitalization from 1998 to 2005 (logit slope = 1.018) became a significantly greater annual increase of 5.1% between 2006 and 2009 .A 1.8% annual increase in the number of involuntary medications After the Dutch governmental initiative in 2006, the earlier rise in numbers of seclusions was successfully changed into a decrease of 2% annually. A reduction occurred despite a rise in the number of involuntary hospitalizations between 1998 and 2009. However, this failed to meet the national goal of reducing the use of seclusion by 10% annually.It may be that the decrease in seclusions could have been larger when the criteria for receiving the grant would have included specific guidelines. For example, despite the evidence on successful interventions for reducing the use of seclusion ,7, thereThe faster increase in the use of involuntary medication after the government’s 2006 initiative was apparent only after correcting for the number of involuntary hospitalizations. Three possible explanations are 1) lower use of seclusion may have led to an increase in involuntary medication, 2) registration was better – which was a criterion for obtaining a seclusion-reduction grant or 3) the increase may have been an effect of a change in the Mental Health Act in 2008. Beforehand, involuntary treatment was only legally allowed to prevent ‘serious danger’ caused by psychiatric illness. In the new legal text the word ‘serious’ was deleted, which broadened the options for involuntary treatment.Although the legal and regulatory context differs between countries, it is interesting to compare our results to other national studies about reducing seclusion. In the USA, for example, after state-wide policy changes, state leadership and other interventions, the Pennsylvania state hospital system successfully reduced the use of seclusion and restraint nearly to zero . SimilarDue to the observational design of this study, we cannot say whether the changes observed in use of the coercive measures were caused by the governmental initiative.The use of seclusion and involuntary medication may have been underreported to the DHCI , especiaSince no data were available on the duration of seclusion, we could not assess the effects of the nationwide program on the length of seclusion.The definition of involuntary medication leaves some room for interpretation, since sometimes the border between persuasion and coercion may be floating. This may have caused underreporting of the use of involuntary medication.Finally, only four years of follow-up were available. A longer follow-up period may be needed to detect longer-term changes in the use of coercive measures.After the start of the nationwide program the numbers of seclusion reduced by 2% annually, a modest reduction that did not meet the goal of a 10% annual decrease. The number of involuntary medications did not change; instead, after correction for the number of involuntary hospitalizations, it increased. These results may be underlain by the absence of national guidelines for reducing coercive interventions.Fur future nationwide programs we recommend use of more extensive national guidelines, especially in implementing and monitoring evidence based programs for reducing the use of seclusion and other coercive measures, and involvement of all relevant wards nationwide.DHCI: Dutch Health Care Inspectorate.The authors declare that they have no competing interests.The conception, design, and interpretation later on, was performed by CM, HN, EN and FV. CM contacted the Dutch Health Care Inspectorate for providing the data. EN provided information on the nationwide program, as he is involved as head of research of this program. FV and EN restructured the data and performed the analyses together with the statistician, DU. DU advised closely on the appropriate method. FV wrote the first and following drafts, while CM, HN and EN revised the drafts critically for important intellectual content. CM primarily re-edited most of the drafts. All authors approved the final version.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/12/231/prepub


We use economy-wide simulation methods to analyze the outcome of a simple REDD+ program in a mixed subsistence/commercial-agriculture economy. Alternative scenarios help trace REDD+’s causal chain, revealing how trade-offs between the program’s public and private costs and benefits determine its effectiveness, efficiency and equity (the 3Es). Scenarios reveal a complex relationship between the 3Es not evident in more aggregate analyses. Setting aside land as a carbon sink always influences the productivity of agriculture and its supply of non-market goods and services; but the overall returns to land and labor–which ultimately determine the opportunity cost of enrollment, the price of carbon and the distribution of gains and losses–depend on local conditions. In the study area, market-oriented landowners could enroll 30% of local land into a cost-effective program, but local subsistence demands would raise their opportunity costs as REDD+ unfurls, increasing the marginal cost of carbon. A combination of rent and wage changes would create net costs for most private stakeholders, including program participants. Increasing carbon prices undermines the program’s efficiency without solving its inequities; expanding the program reduces inefficiencies but increases private costs with only minor improvements in equity. A program that prevents job losses could be the best option, but its efficiency compared to direct compensation could depend on program scale. Overall, neither the cost nor the 3Es of alternative REDD+ programs can be assessed without accounting for local demand for subsistence goods and services. In the context of Mexico’s tropical highlands, a moderate-sized REDD+ program could at best have no net impact on rural households. REDD+ mechanisms should avoid general formulas by giving local authorities the necessary flexibility to address the trade-offs involved. National programs themselves should remain flexible enough to adjust for spatially and temporally changing contexts. International efforts to compensate developing countries for reducing carbon emissions from forests, an initiative known as Reducing Emissions from Deforestation and Forest Degradation (REDD+), have focused most recently on developing implementation mechanisms at the national and sub-national levels In principle, previous experience could provide a basis to design the ideal REDD+ program In this paper we use general equilibrium simulation methods to analyze the direct and indirect effects of a simple REDD+ program and the trade-offs arising between its 3Es at the village level. We simulate several variations on a basic scenario to assess the extent to which alternative reforms deviate from an ideal Pareto outcome . Simulations are designed to highlight the effect of program design given a particular local context, leaving the role of context to be discussed in a subsequent paper. The particular context of this paper–i.e., a Mexican highland village with a mixed subsistence/commercial agricultural sector–allows us to address a pressing but largely overlooked issue: the potential interaction between REDD+ and food production in vast areas of the developing tropics where subsistence agriculture is the norm. To this end we show how outcomes change after incorporating the value of non-market goods and services provided by subsistence activities. In the remainder of this section and the next we present the theoretical and methodological framework of our work. A subsequent section describes the policy scenario and its expected outcomes. A concluding section considers the options available to policy makers charged with designing REDD+ while reflecting on the robustness and generality of our results.Studies modeling REDD+’s causal chain have linked the potential outcome of international negotiations  to land use decisions at a very fine scale in order to develop mechanisms to implement REDD+ across the globe, design incentives for national governments to join or to assess the indirect impacts on global commodity markets A recent study addressing the sub-national level considers the potential distribution of costs and benefits between national and district governments Q’) as the point where the net public benefits of REDD+ are highest. Clearly, both axes in this figure represent single dimensions of a more complex policy problem set within a heterogeneous socio-economic, political and environmental context A common assumption of most previous studies has been that the decisions of national planners and landowners/users coincide. That is, that the social benefits of REDD+ are greatest when the net revenue derived from the land is maximized. This is not necessarily the case. Designing the ideal REDD+ program would require anticipating the full distribution of gains and losses that might follow implementation. In theory, given sufficient data, it should be possible to graph REDD+’s public costs and benefits as a function of specific program characteristics, as we do hypothetically in Q*) solution–one that falls short of public goals but maximizes social  benefits, as shown in Accommodating private interests may require attending to program characteristics not considered up to this point, e.g., the equity of its distribution of payments. Characteristics with both public and private implications could create trade-offs, forcing authorities to settle for a compromise  available to program authorities. This is clearly the case of incentive-based mechanisms such as Payments for Environmental Services (PES), which give flexibility to potential participants rather than to authorities So far we have also referred to REDD+’s outcome as a one-dimensional variable–i.e., monetary costs and benefits–determined by exogenous drivers, but this too need not be the case. Official estimates of the “social cost of carbon” often reduce to a single monetary value Carbon-management programs could have multiple effects on the value and productivity of rural assets, which is the crucial link in REDD+’s causal chain. In the long run, REDD+’s effects could depend on whether interventions create incentives for investment and innovation Mapping the effects of specific program characteristics on productive assets  places uOther authors have relied on analytical methods to decipher the linkages described in While not a substitute for analytical approaches, simulation methods are much better equipped to handle this complexity. Agent-based models, in particular, can help unravel the potential intricacies of REDD+ policy, which has been characterized as a multi-level PES scheme for carbon management Our methodological framework integrates multiple household models into a single model of a local economy in a developing area. The model  introduce alternative reforms that administrators might implement Simulations assume no eligibility requirements other than to set aside any amount of land; i.e., participants can pool resources to avoid minimum-area requirements and reduce fixed costs that may otherwise constrain small holders’ participation The first scenario considers the introduction of a program that seeks to achieve its goal at a minimum cost to the public. The goal is to sequester an amount of carbon equivalent to allowing forest cover to return to 10% of arable land in the locality. Program administrators set carbon prices by calculating the equivalent per-area payment that will secure the level of enrollment desired–i.e., landholders’ “willingness to accept” a payment in exchange for forfeiting the right to use the land Setting land aside has economy-wide repercussions. Agricultural output declines by 3.2%, and given that the program does not hire workers that are laid-off in the process, local wages drop by 2.5% (restoring labor-market equilibrium) . The aveceteris paribus, program participants are not better off than non-participants  losses. This would entail a 23% increase in the program’s public costs (without considering any administrative costs) . A more This scenario simulates a gradual increase in the price of carbon up to 40% above the original price. Such an increase is equivalent to raising per-area payments from 7% up to 50% in excess of the original market rents, which could entice a greater number of landholders to devote larger areas to carbon sequestration. In this scenario, however, the amount of carbon sequestered remains unchanged, so the program’s total area is kept constant and enrollment open to current participants only.  That is, as prices increase, participants in the first scenario must decide whether to leave or remain in the program and how much land to enroll, provided that the total does not exceed the set target.Unsurprisingly, all participants remain in the program, but many reduce their enrollment just slightly as carbon prices rise , col. b.As expected by design, program changes have their greatest impact on wellbeing. Average income losses in the locality diminish gradually as carbon prices rise, and they become gains when prices reach a level 33% above market rents . At pricIn sum, higher carbon prices are highly inefficient as a form of compensation, reducing individual agents’ losses by only 23%  . OverallAs mentioned earlier, another option would be to allow wider participation while generating greater public benefits. There are two possible courses of action for program administrators. If payments per unit area already exceed land rental rates (which entails economic rents), the program’s target can be raised without raising carbon prices. In fact, economic rents can be defined as a payment in excess of what is necessary to secure enrollment, so their existence guarantees an abundant supply of land to the program. On the other hand, if economic rents are nil, increasing enrollment will require increasing carbon prices. We analyze the first option next and report the second option in scenario 1c (see Supporting Information).We now assume that having realized the inefficiency of raising carbon prices, program administrators change course half-way through the previous reform. That is, after raising prices to the equivalent of a payment 14% in excess of market rents while holding the enrollment target constant, they now raise this target gradually  without further price increases. Since the price of carbon is still enticing–i.e., participants initially collect economic rents equal to 6.5% of current rental rates–the number of participants increases from 40 to 52% of local households by the time the program’s area doubles , public costs and benefits both double. Expanding the program also entails a 112% increase in net private costs, which now amount to $103,300 in real terms . SubsistExpanding the program helps transfer participants’ gains to landowners in general, redistributing the gross benefits of carbon management more equitably; but the outcome still is unfavorable for most stakeholders . HousehoOverall, the changes implemented have a mixed outcome. In terms of efficiency, the program now generates twice as many public benefits as in scenario 1a but only 52% more costs. In terms of equity, reforms reduce inequalities between participants and non-participants but fail to improve local livelihoods. Changes mostly transfer the benefits from the largest participants to the largest landowners  and the costs from the general public to local households. Efficiency is restored at the expense of local stakeholders. Avoiding economic rents completely requires removing price controls and constraints on participation simultaneously, that is, achieving the program’s enrollment target via supply-and-demand equilibrium. Scenario 1c contemplates program changes along these lines, revealing the costs and benefits of expanding a cost-effective program (see Supporting Information).All scenarios up to this point are based on the assumption that wages are flexible. Since wages tend to decrease, this assumption allows farmers in the model to increase their use of labor. The assumption is justified only if labor markets are small or relatively isolated, e.g., when workers have few short-term opportunities outside the farm. Our final scenario relaxes the flexible-wage assumption to analyze the case when wages are unchanged after an intervention.We begin this last scenario by simulating a program that offers competitive prices to enroll 10% of arable land, as in the first scenario, except that opportunities for off-farm employment are now widely available , which keeps wages constant. Then we expand enrollment from 10 to up to 20% of land .Constant wages have two opposite effects on both local demand for land and the opportunity costs of enrollment that have implications for carbon prices and program participation. First, constant wages discourage the use of farm labor (relative to previous scenarios), decreasing the intensity of labor (per unit area) and hence the productivity of land. The lower returns discourage demand for arable land and reduce the opportunity cost of enrolling in PES. In such circumstances, program administrators can offer prices only 3.7% higher than market rental rates (compared to 7% in the first scenario) and still reach its target , col. e.Constant wages also have important implications on the local economy, particularly agricultural output, which declines by 8.3% or almost thrice as much as in scenario 1 . SubsistIn sum, constant wages diminish the productivity of land while increasing the value of subsistence output. The effect of these opposing forces on the returns to land–which ultimately determine the opportunity cost of enrollment into PES, the price of carbon and the distribution of gains and losses–is an empirical question. In our model economy, constant wages have a much stronger effect on production than consumption, and so rents are considerably lower than when wages are flexible. Constant wages thus reduce the private costs of PES more than its benefits and redistribute both more equitably. Significant net private losses (in scenario 1) now become net gains of $2,900 in real terms . ConstanThe previous estimate suggests that if off-farm employment opportunities were few (as in scenario 1), the program could employ laid-off farm workers (instead of compensating them for their losses) and still reduce its public costs by 6% . By becoSimulation results show that both production and consumption effects persist when the program covers 20% of the land, but their relative weights change , col. f.Other economic repercussions also are significant. Agriculture contracts up to 10%, commercial output and market surpluses decline up to 32 and 57%, respectively, and food purchases in the open market increase up to 10%–the greatest recorded. The distribution of costs and benefits also differs markedly from flexible-wage scenarios . While wExpanding the program requires substantial resources, raising the public costs to $507,100–or 1% more than under flexible wages . ProvidiTo assess whether the study area is representative of other areas in Mexico, we focus on two conditions crucial to our theory of change: 1) the distribution of productive assets among local households and absentee landowners; and 2) the persistent allocation of these assets to subsistence agriculture. That is, the question is whether we can justify modeling an “income effect” on subsistence production but not an analogous response from commercial producers facing credit restrictions.Landholding sizes vary considerably throughout Mexico, with large farms often interspersing with more numerous smallholdings If efficiency is a goal for REDD+, market-based mechanisms are probably the best alternative Assessing the 3Es of alternative REDD+ programs requires accounting for this context. At the aggregate level, there is a clear trade-off between the efficiency and equity of PES: as carbon prices rise, a program’s private gains increase in direct proportion to its public costs. However, this relationship blurs when the distribution of costs and benefits is considered. In our model economy, raising carbon prices creates substantial benefits for some participants, but it does not compensate most agents’ losses . As discGiven the difficulty of balancing these multiple goals, it should not surprise that the 3Es of PES programs in Mexico have been called into question Our theory of change posits that it is ultimately these three sources of value–wages, rental rates and economic rents–that determine the incentives and social outcome of the various alternatives , top rowIs there an ideal REDD+ program that is effective, efficient and equitable? Our results show that when the diversity of local actors is taken into account, the relationship between the 3Es is complex and characterized by numerous trade-offs. Choosing a second-best alternative requires normative judgments and a precise knowledge of the trade-offs involved. A program that minimizes its public costs could entail significant private losses. Improving the program’s equity while preserving its effectiveness would require transforming those losses into public costs. Whatever mechanism is adopted at the international level, it should avoid general formulas by giving local authorities the necessary flexibility–e.g. in terms of scope, form and amount of compensation–to address these trade-offs. The best option will depend on local conditions, so national programs themselves should remain flexible enough to adjust for spatially and temporally changing contexts.Setting land aside as a carbon sink entails changes in the productivity of land and, whenever subsistence households are present, the value of agricultural output. Its effects on the returns to land could determine the opportunity cost of enrolling land in REDD+ and therefore influence carbon prices, program areas and the distribution of gains and losses. However, given that these effects have opposite signs, their outcome will change from place to place. In the tropical highlands of Mexico, expanding a program or raising the price paid for carbon services could fail to benefit most rural households, particularly subsistence households or those whose main asset is labor. A program that prevents job losses might be the best option, but its efficiency–e.g., compared to direct compensation–could depend on the program’s scale and the extent of demand for agriculture’s non-market benefits in the locality.Our conclusions apply mainly to the densely-populated highlands, but the processes described could occur more widely. Ideally, then, REDD+ should be conceived as a place-based policy, but this assumes that tailoring programs to local conditions will not result in a loss of accountability. Also, authorities would need sufficient knowledge of these conditions to identify the most suitable option. Our methodological framework can be applied at much wider scales provided adequately representative data is available or collected Supporting Information S1(DOC)Click here for additional data file.


A new method based on cloud-point extraction has been developed, optimized and validated for the determination of ergosterol in rat plasma, urine and faeces by liquid chromatography. The non-ionic surfactant Triton X-114 was chosen as the extract solvent. The chromatographic separation was performed on an Inertsil ODS-3 analytical column with a mobile phase consisting of methanol and water  at a flow rate of 1 mL/min. The methodology was validated completely. The results indicated good performance in terms of specificity, linearity, detection and quantification limits, precision and accuracy. The method was successfully applied to the pharmacokinetic studies of ergosterol in rats. The results indicate that the ergosterol levels in feces are much higher than those in plasma and urine of the rat.Ergosterol from many medicinal fungi has been demonstrated to possess a variety of pharmacological activities  Polyporus umbellatus, Cordyceps sinensis, and Hypsizigus marmoreus [Polyporus umbellatus [in vivo might be influenced by many factors, such as body-and/or cell-compartment distribution, drug metabolism, lipophilicity, membrane permeability, and protein binding. So, these multiple pharmacological activities of ergosterol make it worth carrying out further study on pharmacokinetic properties and elimination pathway of ergosterol.Ergosterol is one of the best-known steroids, which exists widely in many medicinal fungi such as armoreus –3. We habellatus . Ergostebellatus  and antiA number of methods have been reported for the quantification of ergosterol in raw materials, such as high performance liquid chromatography-ultraviolet detection (HPLC-UV) , 6–11, hPolyporus umbellatus. The procedure isolation and purification of ergosterol was published in our previous paper [1H NMR, 13C NMR, and MS) and compared with those found in the literature [Polyporus umbellatus purified in our laboratory with 99% purity as determined by HPLC. HPLC-grade methanol was purchased from Baker Company . Ultrahigh purity water was prepared by a Millipore-Q SAS 67120 MOLSHEIM (France).The standard of ergosterol  was isolus paper . Its strterature . Ergostaµg/mL) and ergone  were prepared by dissolving suitable amounts of pure substance in acetone and were stored in darkness at 4°C.Standard stock solutions of ergosterol  samples were separately prepared in a similar manner as those used for the calibration curve. Concentrations of 0.25, 0.75, and 2.00 µg/mL were used for plasma and urine calibration standards, whereas concentrations of 0.25, 25, and 100 µg/mL were used for faecal homogenate corresponding to the low QC, medium QC, and high QC, respectively.Faecal samples were homogenized with acetone in the ratio 1 : 10 (g : vol = faeces : acetone) to obtain faecal homogenate. For HPLC determination, calibration standards of ergosterol at concentration levels of 0.09, 0.25, 0.50, 0.75, 1.00, 2.00, and 2.50 µm, Japan) with the column temperature set at 30°C. An isocratic elution of methanol (A) and water (B) was 98 : 2 (v/v) in 23 min. The flow rate was 1.0 mL/min with detector wavelength set at 283 nm (maximum absorption wavelength of ergosterol), and the injection volume was 20 µL.HPLC was performed with a Waters 2695 instrument composed of a quaternary pump, a column oven, and a Waters 2487 dual wavelength absorbance detector, and Empower was used for data collection. The chromatographic separation was performed on an Inertsil ODS-3 analytical column  of IS working solution and 135 µL of 0.3 mol/L sodium chloride solution. The contents were mixed for 2 min, and then followed by the addition of 100 µL of 5% Triton X-114 (v/v) aqueous solution. After that, the obtained contents were well mixed again for 5 min and then incubated in the thermostatic bath at 40°C for 20 min. After the phase separation was formed by centrifugation at 5000 rpm for 10 min, the surfactant-rich phase was obtained in the bottom of the tube by the removal of the water phase. 100 µL of mobile phase was spiked to the surfactant-rich phase. Then, the contents were mixed and centrifuged at 16,000 rpm for 5 min, respectively. Most of the surfactants and coextractants such as hydrophobic proteins were precipitated in the bottom of the tube, and 20 µL of supernatant fluid was injected into the HPLC system for analysis.The plasma or urine was prepared by a cloud-point extraction method. In a 1.5 mL capped centrifugal tube, 50 µL volume of faecal homogenate standard or sample was transferred to a 1.5 mL centrifuge tube, and then 30 µL of IS working solution was spiked and vortex-mixed for 1 min. 70 µL of acetone was added to the solution and vortex again for 2 min. The mixture was centrifuged at 10,000 ×g for 10 min. A 20 µL of supernatant of the solution was injected into the HPLC for analysis.A 100 µg/mL) and calibration standards for faecal homogenate  were extracted and assayed. The linearity of the calibration curve was confirmed by plotting the peak-area ratios of ergosterol to IS versus the ergosterol concentrations with a 1/x-weighted least-squares linear regression analysis. The limit of detection (LOD) was considered as the final concentration producing a signal-to-noise ratio of 3. The lower limit of quantification (LLOQ) was considered as the lowest concentration on the calibration curve where precision was within 20% and accuracy was within 100 ± 20% [ HPLC-UV method was applied for the quantification of ergosterol from rat plasma, urine, and faeces. Calibration standards of seven concentrations of ergosterol for plasma and urine  thaw (room temperature) cycles.To validate the method with real samples, a trial was undertaken to determine ergosterol in the plasma and excretion samples of healthy rats, which were administered a single dose of ergosterol orally. The study was conducted in accordance with the Regulations of Experimental Animal Administration issued by the State Committee of Science and Technology of China. All procedures and the care of the rats were in accordance with institutional guidelines for animal use in research. Rats were housed in individual metabolic cages on standard laboratory food bedding in a temperature controlled room (22 ± 2°C) with a 12 h light/dark cycle. n = 6) based on the time of blood sampling with two animals each. The control groups received the vehicle only. The blood samples (approximately 300 µL) were collected from vena orbitalis in heparinized tubes at control and 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 16, 24, and 36 h after the administration. Samples were immediately centrifuged at 5,000 ×g for 10 min and the plasma was frozen at −20°C and stored until analysis. The estimation of ergosterol in all the samples was undertaken within 36 h of blood collection by the method described previously. Male Sprague-Dawley (SD) rats, 200 ± 10 g, fasted overnight with free access to water for at least 12 h, were dosed orally by gavages with 100 mg/kg body weight of ergosterol dissolved in plant oil as a vehicle. Rats were divided into three groups (n = 6) based on the time of urine and faeces sampling with three animals each. The control groups received the vehicle only. Rats were housed with free access to food and water in individual metabolic cages, except for the final 12 h before a single oral administration of 100 mg/kg of ergosterol (access to water was ad libitum during the experiment). Faeces and urine were collected after administration in different periods . The amount of faeces and urine collected over each period was recorded, respectively, and then urine and faeces were stored at −20°C until analysis.Excretion was studied in another four groups  and pharmaceutical parameters were obtained.The percentage of ergosterol eliminated in faeces was calculated with the accumulated ergosterol eliminated in all periods divided by administration amount .1H NMR  3.63 , 5.39 , 5.58 , 0.63 , 0.95 , 1.04 , 5.18 , 5.22 , 0.83 , 0.84 , 0.92 ; 13C NMR  38.3 (C-1), 32.1 (C-2), 70.6 (C-3), 40.8 (C-4), 139.9 (C-5), 119.7 (C-6), 116.4 (C-7), 141.5 (C-8), 46.4 (C-9), 37.1 (C-10), 21.3 (C-11), 39.0 (C-12), 42.8 (C-13), 54.7 (C-14), 23.1 (C-15), 28.4 (C-16), 55.8 (C-17), 12.2 (C-18), 17.7 (C-19), 40.5 (C-20), 21.2 (C-21), 135.5 (C-22), 132.1 (C-23), 42.9 (C-24), 33.2 (C-25), 19.8 (C-26), 20.1 (C-27), 16.4 (C-28). The data of mass spectrum was published in our previous paper [The characteristics of ergosterol are  CDCl3, 1 MHz 38.3µm or Diamonsil C18 column, 250 mm × 4.6 mm i.d., 5 µm), temperature (30 or 35°C), and peak shape. The test results showed that the peak shapes of ergosterol were improved by the solvent system of methanol and water.Chromatographic optimisation of HPLC conditions was carried out with respect to mobile phase conditions (methanol-water or acetonitrile-water), stationary phase  from blank rats, each as a blank sample and a spiked sample. The chromatograms of ergosterol and the IS are showed in Figures µg/mL for plasma and urine and 0.08–125 µg/mL for faecal homogenate. The typical equation was Y = 0.8251  X + 0.0357 (r = 0.9996) for plasma, Y = 2.9189  X + 0.1265 (r = 0.9995) for urine, and Y = 0.0262  X + 0.0104 (r = 0.9994) for faecal homogenate. Y = peak-area ratio (ergosterol/IS) and X = ergosterol concentration. During routine analysis, each analytical run included a set of calibration samples, a set of QC samples, and the unknowns. The LOD of ergosterol in rat plasma, urine, and faecal homogenate were 42, 50, and 49 ng/mL, respectively. The LOQ of ergosterol in rat plasma, urine, and faecal homogenate was 90, 90, and 80 ng/mL respectively.Calibration curves of ergosterol were linear over the concentration range of 0.09–2.50 The selectivity of the method was demonstrated by comparing chromatograms of five independent biological samples  from blank rats, each as a blank sample and a spiked sample. n = 5). Recovery data are shown in The mean extraction recovery of ergosterol from plasma, urine, and faecal homogenate was 90.9 ± 2.9%, 92.4 ± 2.6%, and 93.7 ± 3.3%, respectively. The mean relative recovery for IS was 94.5 ± 3.2%  made this method suitable for routine analysis in preclinical pharmacokinetic studies, and the method was helpful in clinical pharmacokinetic studies. It can also be used as a reference for therapeutic drug monitoring of ergosterol.


Health Technology Reassessment (HTR) is a structured, evidence-based assessment of the clinical, social, ethical and economic effects of a technology currently used in the health care system, to inform optimal use of that technology in comparison to its alternatives. Little is known about current international HTR practices. The objective of this research was to summarize experience-based information gathered from international experts on the development, initiation and implementation of a HTR program.A mixed methods approach, using a survey and in-depth interviews, was adopted. The survey covered 8 concepts: prioritization/identification of potentially obsolete technologies; program development; implementation; mitigation; program championing; stakeholder engagement; monitoring; and reinvestment. Members of Health Technology Assessment International (HTAi) and the International Network of Agencies for Health Technology Assessment (INAHTA) formed the sampling frame. Participation was solicited via email and the survey was administered online using SurveyMonkey. Survey results were analyzed using descriptive statistics. To gather more in-depth knowledge, semi-structured interviews were conducted among organizations with active HTR programs. Interview questions were developed using the same 8 concepts. The hour-long interviews were recorded, transcribed and analyzed using constant comparative analysis.Ninety-five individuals responded to the survey: 49 were not discussing HTR, 21 were beginning to discuss HTR, nine were imminently developing a program, and 16 participants had programs and were completing reassessments. The survey results revealed that methods vary widely and that although HTR is a powerful tool, it is currently not being used to its full potential. Of the 16 with active programs, nine agreed to participate in follow-up interviews. Interview participants identified early and extensive stakeholder engagement as the most important factors for success. A lack of top-down support and financial and human resources are inhibiting program development.HTR is in its infancy. Although HTRs are being conducted, there are no standardized approaches. However, much can be learned from current international work. Future work should focus on developing a comprehensive methodology, reporting the processes of reassessments and sharing successes and challenges in a common platform. Health Technology Reassessment (HTR) has been garnering interest over the past decade as a method of improving quality of care and financial sustainability. HTR is a structured, evidence-based assessment of the clinical, economic, ethical, and social effects of a technology currently being used in the health care system, to inform optimal use of that technology in comparison to its alternatives. There aA recent systematic review revealed that there is only one published model for HTR. The sysA mixed methods approach, using a survey and in-depth interviews, was used to gather experiential knowledge. This approach was used in order to acquire an overview of the state of HTR activity internationally while gaining a deeper understanding of the mechanics and practicalities of established HTR programs.A convenience sampling strategy was used. The sampling frame consisted of the organizational and individual members of Health Technology Assessment International (HTAi) and the organizational members of the International Network of Agencies for Health Technology Assessment (INAHTA); the two major international organizations of health technology assessment producers and users. A pre-notification email was sent to this sample seeking participation and requesting the name of a contact within their organization knowledgeable in HTR. Six days subsequently, an email containing the survey link was sent out. This link went out to the contacts who were identified as being involved in HTR and to the individuals who did not respond to the pre-notification email. The survey questions were developed to obtain information on eight broad areas that were defined a priori from a systematic review of the literature, as important facets of the HTR process: identification and prioritization of potentially obsolete technologies; program development; implementation; mitigation; program championing; stakeholder engagement; monitoring; and reinvestment. An online tool, SurveyMonkey, was used to conduct the survey from October 24th – November 7th, 2011.The online survey was designed to stream participants into one of three sets of questions based on the organization’s stage of program development  of the survey respondents identified themselves as either researchers or senior managers. Nearly 34% responded that their role fit into an ‘other’ category, which included economists, professors, students, pharmacists and research fellows.Nineteen of the 49 respondents who were not considering developing a HTR program cited that reassessment was not within their mandate. Seventeen of these nineteen respondents said that they had insufficient resources to develop the program, such as inadequate funding, time and qualified personnel, as the reason for not developing a HTR program.Those with established or developing programs were asked why their organization decided to invest in the development of a HTR program Figure . When thIn the survey, participants identified barriers they had encountered during development and implementation. Those still in the development stage responded that the top three barriers were: a lack of expertise in HTR (n = 10); political barriers (n = 9); and a lack of interest (n = 9). Those who have an active program cited political barriers (n = 10) as the most significant hurdle followed by a lack of expertise in the field (n = 6).Nine of the sixteen individuals with active programs noted that their organization is responsible for mitigating barriers that arise during the HTR implementation process. Survey results suggest that most organizations use championing and stakeholder engagement as primary means of barrier mitigation. Common types of stakeholder engagement include: involving clinicians to increase buy-in; and having public representatives involved in the process to increase knowledge of the HTR process.Of those organizations actively doing health technology reassessments, 11 have a champion for the program Figure . Of thesParticipants with established or developing programs (n = 25) were asked how recommendations or decisions to reduce or remove funding for a technology would be implemented. Fifty-four percent of the organizations developing a program responded that they only have advisory capabilities; and therefore, the final reassessment decisions lie in the hands of an external decision-maker. Less than five percent responded that their organization had the ability to directly implement a HTR decision. Fifty percent of those who were part of an established HTR program said that they had advisory capabilities, whereas over 20% noted that they controlled funding decisions and therefore could implement their HTR decisions.Participants who had established or developing programs, were also asked “If funds are removed from a technology that is reassessed, what happens to the resources that were allocated to it?” Four of the respondents with an established program noted that after disinvestment, funds are reallocated to a more efficacious technology; of those with a program in the development stage, eight noted that the funds were reallocated to a more efficacious technology. The survey results also revealed that many individuals involved in reassessment do not know what happens to funds that are liberated. Six organizations with developing programs and three with established programs were unsure of what would happen with liberated funds.The sixteen survey respondents who indicated that their organization had an active reassessment program were invited to participate in a follow-up interview; nine indicated interest. These nine interview participants came from six jurisdictions: Australia, UK, Canada, Sweden, Costa Rica, and Italy; and consisted of policy decision-makers, researchers, senior managers, and government officials. As described previously, the purpose of these interviews was to obtain in-depth perspectives on the key questions asked in the survey.With respect to reasons for initiating a HTR program, participants affiliated with academic settings generally noted that their HTR program was established with the goal of developing robust HTR models with rigorous economic evaluation components. Those affiliated with government tended to see HTR as a promising approach to improve fiscal sustainability in health care. At the health care delivery level, HTR was described as being a tool for advancing evidence-based practice while improving health care quality and safety. The increasing budgetary pressure facing governments and health care systems was also described as a key factor prompting the development of HTR programs.“…lots of people do little projects that essentially have an element of disinvestment in them, but nobody is using that terminology.” Organizations also conducted reassessments in varying degrees of depth. Some described taking a broader look at evidence-based practices (using both HTA and HTR), for example, reassessing all of the technologies from an entire clinical issue or area, whereas other organizations described reassessing one particular procedure or technology at a time.Throughout the interviews, it became clear that there are many methods for completing reassessments, with little standardization internationally. Some interviewees explained that they did not have a separate HTR program, but rather had integrated HTR into their HTA work. One person working in a unit at the health service delivery level stated that: The interview participants all encountered significant barriers while implementing their HTR programs. Some of the key barriers frequently cited included a poor evidence base, political push-back, a large investment of work and time for relatively small cost-savings, influence of industry, and difficulty communicating with a variety of audiences. Many methods for mitigating these barriers were proposed with the top two being stakeholder engagement and champion involvement.…is potentially controversial and often, ideologically not well aligned with political interests, so having a political champion who is receptive…is important.”Having a champion involved who can provide top-down support emerged from the interviews as a key strategy for developing and implementing a successful HTR program. This finding echoes the survey results presented earlier. During the interviews, one resarcher noted that HTR: "“We take them on the journey with us.” To this end, interview participants spoke at length about ways of engaging stakeholders; including the importance of presenting the reassessment findings in a way that is comprehensible and intuitive to a diverse health professional and public audience. Many approaches were highlighted in these conversations. Integrating reassessment findings, along with other kinds of evidence-based guidance, into care pathways was often brought up as a successful strategy, as this quote illustrates: “The clinicians and even patients…found that the most interactive and the most easily understood [approach to presenting the guidance] was the pathway of care they needed to follow… Looking at all the appraisals and guidelines…they all need to feed into the same clinical area…[we have determined] that would be the way to best present our information.”Interview participants described stakeholder engagement as necessary for success, noting that it was crucial to engage stakeholders promptly and continuously to be successful with HTR. As one senior government official commented: Participants were also asked to describe the model, or framework, they used to complete their reassessments. Most frequently, the HTR models being used were developed by those delivering the HTR programs. It was regularly stressed that models were highly context-dependent , meaning models developed elsewhere were difficult to adopt without modifications. None of the interviewees indicated that they had used a model developed elsewhere. Interview participants also emphasized that models and HTR processes tend to evolve over time based on successes and failures; so having a flexible model was imperative.Although models and corresponding processes were developed to fit local context, many models do incorporate many of the same broad stages or steps. Most begin with a new or existing committee identifying and prioritizing potentially obsolete technologies for reassessment, while beginning to engage potential stakeholders and clinical experts. Prioritization is often based on some combination of utilization rates, ease of reassessment, mechanisms for change and potential cost savings. A review is then conducted on the selected technology and a recommendation or decision is put forth based on the collected evidence and input from stakeholders. The results of a reassessment are often made available through public websites, news releases and\or academic publications.Since most final funding decisions are made at the governmental level, successful HTR implementation relies on processes outside the control of the organization conducting the review. One interviewee noted that approximately 50% of the time, the decision-making body accepted the recommendations coming from the review and advised the government to implement the decision. In this case, the government almost always acted on the Board’s advice. This rate of concordance reflects the political and social influences on decision-making.…trying to measure that without a health economist expertise is going to be really difficult. I think we may not be able to quantify the savings.”Interview data suggests that reinvestment strategies are either lacking or unknown. This may be because monitoring processes are not in place and/or because reallocation decisions are beyond the scope of the organization’s mandate. Whether money saved can be reallocated back to the same clinical area toward a more effective technology is dependent on the type of savings and how budgets are allocated. In health care delivery organizations, for example, sometimes the savings occur in areas that are not aligned with a particular clinical department . One interview participant noted that it would be difficult to keep track of cost-savings: "Results of this research indicate that many organizations realize the potential benefits of HTR and are interested in integrating it into their health care system. However, a lack of top-down support, push back from clinicians, financial and human resource limitations, and a lack of expertise in health re-assessment are inhibiting program development.Those with developed programs offered sage advice for successful program development and implementation. It was often reiterated by interview participants that when launching a program, it is ideal to select technologies where the reassessment process is unlikely to be lengthy and will result in significant fund liberation. This will increase the likelihood that public understanding of HTR will be improved and get relevant stakeholders on board.The importance of learning from successes and failures, and having the flexibility to adjust models and processes accordingly was frequently highlighted. Similarly, it was continually reiterated that communication with stakeholders is critical to the success of a HTR program. Stakeholders must be engaged early, continuously and as extensively as resources allow. With implementation, it is important to have people and processes in place to move a decision into practice and policy.There is no commonly accepted model for reassessing health technologies and very little agreement on specific methods amongst those completing HTRs. Although interview participants revealed that models must be very context-dependent, having even a broad outline of steps and stages based on effective HTR programs may help develop more successful programs.Such a model must include reinvestment and monitoring; two elements infrequently incorporated into current models. In the development of these stages, international collaboration and perspectives would increase the applicability of such a model. There are a few experts who possess a wealth of experience-based knowledge on the subject. Since there are few people who have in depth knowledge of this field, many people indicated that not having expert knowledge within their organization was preventing them from developing a HTR program. For this field to develop to its full potential, an emphasis on communication and collaboration between interested organizations is essential. Focusing on using collaboration and knowledge-sharing as a means to move the HTR agenda forward has been emphasized in current literature-8.This study has limitations. The survey response rate was low; as this is a new and small field, this was anticipated. The sampling frame was intentionally broad and over-inclusive to ensure that all potential respondents were captured. The number of key-informants was small. However, many of the same themes were repeated across the interviews, indicating that we were close to achieving saturation in the thematic areas reported here.HTR has the potential to improve the quality of care patients receive, promote evidence-based practice and improve health care sustainability. It is a powerful tool that is not being used to its full potential internationally. However, it is currently limited by an under-developed theoretical base, a small field of experts and limited reporting of case studies in the literature. Future work should focus on advancing the theoretical framework for HTR drawing upon developed theories in knowledge translation and implementation science. In addition, case reports of HTR applications that include a strong monitoring and evaluation component should be reported in the scientific literature. This will contribute to the development of a body of knowledge from practice that has great potential to improve our understanding of what works and what does not work in which contexts and why, that those embarking upon HTR could draw upon.HTR: Health technology reassessment; HTA: Health technology assessment; HTAi: Health technology assessment international; INAHTA: International network of agencies for health technology assessment.None of the authors have competing interests to disclose.Design of the study LL, TN, LS, GM, FC; collection of data LL, GM, TN FC; management of data LL, GM, analysis of data LL, GM, FC; interpretation of the data LL, GM, FC; preparation of manuscript LL, FC; review of manuscript LL, GM, TN, LS, FC; approval of manuscript LL, GM, TN, LS, FC. All authors read and approved the final manuscript.Alberta Health. Survey Questions.Click here for file