Citizen science programs are increasingly popular for a variety of reasons, from public education to new opportunities for data collection. The literature published in scientific journals resulting from these projects represents a particular perspective on the process. These articles often conclude with recommendations for increasing “success”. This study compared these recommendations to those elicited during interviews with program coordinators for programs within the United States. From this comparison, success cannot be unilaterally defined and therefore recommendations vary by perspective on success. Program coordinators tended to have more locally-tailored recommendations specific to particular aspects of their program mission. Science is a process that cannot be completely understood without fully understanding the actors, their interactions with each other and nature, and their context The recent upswing in both quantity and scale of citizen science projects can be attributed to an increase in enabling technologies, the need for large-scale datasets, and the push for outreach by funding agencies The science studies literature describes these benefits collectively as resulting from recognizing the value of non-scientific expertise . Citizens experience the world in ways outside the traditional scientific method, allowing physical and temporal access to the question at hand that is necessary to provide these benefits and others. The perspectives citizens bring from their individual ways of knowing can offer creative approaches to scientific problem-solving and connect scientific information to policy applications The focus on process by citizen science is a small-scale example of a larger societal phenomenon recognizing many forms of expertise in science – termed “democratizing science” The benefits of citizen science and democratizing science originate from emphasizing process – specifically by spreading power derived from expertise among more people, creating a scientifically educated public for future scientific development, and connecting science to everyday life. Citizen scientific co-production of knowledge parallels benefits from co-production of policy, or co-management Theorizing knowledge co-production and nesting citizen science within the broader movement to democratize science are recent phenomena. Many citizen science programs pre-date scholarship on the matter, hence there are concerns when attempting to achieve both scientific and social benefits in one program In a fully democratized science, ‘success’ depends not only on the quality of the product but also the process. There is still a demand for the “best available science”, interpreted as high-quality data and building blocks for scientific theory development, especially from the established scientific community Since citizen science efforts predate the movement for a democratized science and therefore may not employ the broadened definition of success, academic literature might leave out citizen recommendations for improving that success. Specifically, published articles favor the hypothesis-testing model of science that many citizen science projects do not easily fit into, suggesting the need for a more thorough review We identified leaders of citizen science projects through a Google search using the search terms ‘volunteer’ and ‘monitoring’, followed by systematic random sampling of the search results beginning on a random page, using programs that identified as citizen science within the United States on their webpage. The search was intentionally structured this way to focus on long-term, environment-based citizen science programs. This left out other disciplines’ citizen science projects ; of course, we ran the risk of missing programs that did not identify themselves using this particular terminology, but we achieved an adequately sized group of programs for analysis. We contacted the phone number listed for either the program coordinator or primary investigator, with an overall 50% response rate for a total of 19. The non-responses were all from incorrect contact information: disconnected phone lines while not in field season  or people no longer affiliated with the program. The responses may therefore favor programs with year-round staff and larger programs with up-to-date websites.Structured interviews lasted around 20 minutes each following an interview guide with questions addressing program mission, daily function, and recommendations for other programs to be successful. We coded answers according to code trees directly addressing our research questions, adding sub-codes to each tree as they emerged. We also took notes during the interviews, especially of tangential or explanatory information, for later analysis coded using grounded theory We identified articles for review through a search in three databases covering education (ERIC), environmental science (Web of Science), and sociology . Search terms included ‘citizen science’, ‘volunteer’, ‘assessment’, and ‘monitoring’. We reviewed each study that occurred within the United States and coded them according to the following questions (derived from the interview script for ease of comparison); articles answering none were dropped from review, leaving a total of 67:Did the author consider the study successful as determined by the stated purpose of the study?Did the author consider the data accurate? At what level?What are the recommendations of the authors for other investigators hoping to use citizen science?Program coordinator interviews reached programs that were on average 12 years old, with variation from one to 38 years. Programs size varied, having an average of 279 volunteers ranging from 15 to approximately 5000  collaborate with experts, 2) have a consistent methodology, and 3) present data to policymakers see . AnswersOver half (57%) of the literature used in the meta-analysis reported results from citizen science programs existing for more than five years. Most of the funding for these programs came from either government (30%) or academia (43%), which is often funded through grants from the National Science Foundation. The stated purpose of these articles, with the exception of two, was to aid the development of a citizen science program in a particular location or habitat. These demographic traits mean that most of the articles discussed programs that had long-term history and long-term goals.The top recommendations for the literature are as follows: 1) collaborate with experts, 2) present data to policy influencers, 3) have a consistent methodology, and 4) have a standardized training program see . Each ofAlthough the top recommendations were shared in the literature and surveys, there are a few notable differences. First, the fourth recommendation from the literature – to have a standardized training program – was only mentioned by one survey respondent, who commented that the only reason training was an issue for his program is because they hadn’t instituted standardized training from the inception of the program. Several forest monitoring programs mentioned the need to “get it right the first time”, incorporating the idea that initially setting up the program correctly, on a number of facets, is more important than any one aspect of program function.The survey respondents created a much longer, more varied list of suggestions than the literature review generated, generally tailored to specific challenges their program had faced. Though these results were initially coded to the same categories as those found in the literature review, this specificity is notable because they are written for new program directors to be successful in similar programs. The literature review recommendations were aimed at a more general audience. For example, the top recommendation of the need to collaborate with experts was to “look at other existing programs”, for “networking, the way to answer specific questions”, or as “crucial to pull resources together effectively”. Many groups also stated they “absolutely would not be able to do anything [without partnerships]” and that meeting the other recommendations followed as a consequence of collaboration. These specifics better describe the nature of the collaboration, which experts are involved, and what products are expected as a result.The recommendation of needing consistent methodology was often phrased in the context of a particular challenge the primary investigator faced. For example, some doctorate-holding volunteers in a Gulf of Maine monitoring program felt the need to revise existing protocols on the fly; the program coordinator attributed achieving consistent methodology in this case to enforcement and emphasis on existing written protocols. In contrast, the methodology challenge to most stream monitoring programs is the level of taxonomic detail required to follow the protocol; they instituted field handbooks in response. In yet other cases, practice made perfect, where more than 3 hours of training were required before volunteers could measure and officially record data.The final recommendation from the literature, connecting to policymakers, was more straightforward. Still, there were some groups that approached the citizen science process as objective scientists and felt the data should speak for themselves. In these cases, they wanted to be able to hand over raw data to state or local agencies to include in larger databases used for decision-making. In other cases, the approach was through advocacy, where digested results would be presented at public meetings, annual volunteer meetings with invited policymakers, or included in letters to elected representatives.Both sources of recommendations direct future programs toward behaviors for success. However, not all programs had the same definition of success. The mission of each program was not purely about creating scientific information, but about some combination of education, restoration, stewardship, and community-building. As with the recommendations for success, the surveys revealed more nuanced answers about mission or purpose. Many respondents hesitated a moment before classifying the success of their program, given the mission statement.“Success” also depended on the larger socio-ecological context. One program coordinator stated that it is “hard to measure success” when watching floundering lobster nurseries despite their program’s efforts. She knew that they had saved at least six nurseries from development projects, but still had a hard time classifying the last few years of her program as ‘successful’. Others respond to context by changing their mission. A coordinator doubted the relevance of their mission, expressing uncertainty as to “who we should become next… we’re at a crossroads”. This program could not have success before defining their new mission. One grassland program described the work of the group as on an “as we can handle basis”, shifting from one mission to the next as need arises.Stated program purposes fell into 6 general categories: mapping species distribution, increased stewardship, restoration, baseline data, tracking trends, and “not sure”. Examples of these missions include “create a habitat for the future”, take “just a snapshot”, “actively changing the program based on what the public needs”, and “to get data, but awareness and stewardship”. 84% of coordinators classified their program as having met its purpose with slightly less (68%) stating confidence in their data .Because there are a multitude of purposes, the questions “do you consider yourself successful” and “is your data reliable” do not necessarily address the same thing. Though most coordinators were confident in their data, that confidence came with the caveat that the data may not be professional, but results are certain “from the sheer amount of time out there”. Others recognize that not all their data is reliable but they are “along the right steps” and have some data to share. The extreme ends of the spectrum also existed, creating data that “absolutely and exceeded” certainty or “data, maybe…education, certainly”.U. americana from U. rubra” The articles reviewed claimed success most of the time (97%), with their overall goal being scientific in nature. Part of this result may be due to the tendency for scientists to only publish positive results, leaving negative results to the grey literature or a filing cabinet The concept of community, both inside the program and within the program’s region, was mentioned by many of the respondents in different contexts. When asked about program demographics, many coordinators said a better question to ask is how many people from the community are involved. Especially for programs in small towns, coordinators were proud of the fact that nearly every resident participated in some way in the program. They stated that the community-building process strengthened their program overall. One program held an annual summit, which the coordinator described as putting “context to their work” by presenting efforts to the community and showing volunteers tangible results.At first glance, literature and interviews yielded similar recommendations – but only in the general sense. Interviews revealed a much more varied list of suggestions leading to less clear “top” suggestions (which were reported here for comparison). The recommendations out of the literature employed blanket statements strategizing scientific successes. Such recommendations need to be filtered through the particular context of a program, be that mission priorities, habitat, volunteer demographics, or something else. The more specific suggestions made by program coordinators emphasized particular parts of the process – collaboration for resources or new research questions rather than legitimacy through ties to established experts or data-checking. Thus, recommendations from the literature may be helpful when planning the broad aspects of a program but not as helpful in the first few struggling years that every program goes through.Similarity in these broad-scale recommendations is logical because successful process often leads to successful product – in this case, reliable data. The difference between the recommendations from the literature and the surveys lies in recognizing the mechanism of achieving that end and the concomitant benefits. For instance, presenting data to policymakers may at first seem tangential to the scientific goal of a program, but through raising excitement in the program, such action ensures more long-term volunteers that maintain the strength of the program and the integrity of the data The main difference in the broad-scale recommendations is that the standardized training emphasized in the literature did not emerge as a priority among program coordinators. Standardized training may already be expected in the structure of a volunteer program since multiple people are collecting data to later be compared. Perhaps this is so obvious an expectation that it need not be said in formalized recommendations. Alternatively, the term ‘standardized training’ might imply that volunteers are merely “data monkeys” with little input in their own educational experience. Since volunteers may expect constant interaction with scientists and other volunteers as part of their learning experience – and a common mission of the programs – standardized training may serve only as a minimal educational baseline; therefore it is prioritized in the literature recommendations focused on data, but not by program coordinators who aim for increasing education.The programmatic mission as determined through the phone surveys varied, making the definition of success different for each program. Only one of the intended program purposes (collecting baseline data) was data-focused, defining success by the quality of data collected. Others were process-focused, such as tracking changes over time or increasing stewardship in participants. The process was more often successful than the product (data) and some of the respondents pointed out that the data is “good enough” or not the main focus of the program. This is in stark contrast with the published studies, many of which discuss citizen science as a method, evaluated against traditional methods by the same metrics of success – data quality.Even though most of the programs were confident in the reliability of their data, most had also dismissed at least some of the collected data as unreliable. Program coordinators reported having many still-unpublished results, either because they were negative or not reliable enough. Consequently, the publications resulting from these types of programs represent a self-selected group with enough positive results to write about. The pool of literature found for the review likely represents a similar fraction of programs in existence. These programs had some component of their mission focused on creating baseline data. The recommendations from these programs found in the literature therefore refer to specifics that address only one type of mission, which is not representative of the myriad programs practicing.Success in citizen science programs is defined by the particular mission statement guiding the program, which is more likely to focus on the scientific process than purely on the results. Of course, a strong and successful process is more likely to lead to reliable data, but not necessarily in all cases. The differences in the specificity of recommendations observed between the literature and surveys can be explained by what they were making recommendations for. The literature recommendations represent a subset of projects with successful missions relating to providing data while survey recommendations were more sensitive to other aspects of missions and incorporated more struggling programs. Thus the focus on process should be made explicit when making recommendations, advertising or fundraising for citizen science groups.


On 22 February 2011, Christchurch New Zealand  experienced a devastating earthquake, causing extensive damage and killing one hundred and eighty-five people. The earthquake and aftershocks occurred between the 2009 and 2011 waves of a longitudinal probability sample conducted in New Zealand, enabling us to examine how a natural disaster of this magnitude affected deeply held commitments and global ratings of personal health, depending on earthquake exposure. We first investigated whether the earthquake-affected were more likely to believe in God. Consistent with the Religious Comfort Hypothesis, religious faith increased among the earthquake-affected, despite an overall decline in religious faith elsewhere. This result offers the first population-level demonstration that secular people turn to religion at times of natural crisis. We then examined whether religious affiliation was associated with differences in subjective ratings of personal health. We found no evidence for superior buffering from having religious faith. Among those affected by the earthquake, however, a loss of faith was associated with significant subjective health declines. Those who lost faith elsewhere in the country did not experience similar health declines. Our findings suggest that religious conversion after a natural disaster is unlikely to improve subjective well-being, yet upholding faith might be an important step on the road to recovery. After the wind there was an earthquake, but the Lord was not in the earthquake. After the earthquake came a fire, but the Lord was not in the fire. And after the fire came a gentle whisper. Then a voice said to him, “What are you doing here, Elijah?”1 Kings 19: 11–13W 6.3 earthquake struck Christchurch New Zealand  at a depth of only five kilometres, resulting in New Zealand's most deadly natural disaster in eighty years. Two multi-story office buildings collapsed, igniting fires, killing one hundred and ten. Falling bricks and mortar crushed two city buses, killing six. Another sixteen died from falling debris and landslides. Many hundreds were injured. The Catholic Cathedral of the Blessed Sacrament was ruined. The cross atop its dome was set to lean. The Anglican Cathedral at Christchurch was shattered. Its brick spire, a city landmark, was toppled  This agony was anticipated by a less deadly but larger earthquake on 4 September, 2010 (MW7.1), and aftershocks rattled Christchurch for many months after the February event. All told, roughly a third of the buildings in Christchurch were damaged beyond repair . The New Zealand Treasury estimated the total costs for rebuilding Christchurch to be NZ$ 20 billion, or about US$ 15 billion . Residents were left in doubt for months about the extent of government and private insurance, prolonging their agony.At 12:51pm on 22 February 2011, a M toppled . Emergenhttp://www.psych.auckland.ac.nz/uoa/NZAVS). The timing of the earthquakes between two survey points afforded the unprecedented opportunity to examine how a natural disaster of this magnitude affects deeply held religious commitments and global ratings of personal health.The earthquakes and their aftermath occurred between the 2009 and 2011 waves of the New Zealand Attitudes and Values Study, a large, countrywide longitudinal survey assessing change and stability in values .No one is under the illusion that nature has already been vanquished there are elements, which seem to mock at all human control: the earth, which quakes and is torn apart and buries all human life and its works; water, which deluges and drowns everything in a turmoil; storms, which blow everything before them; there are diseases, which we have only recently recognised as attacks by other organism, and finally there is the painful riddle of death, against which no medicine has yet been found, nor probably will be. With these forces nature rises up against us, majestic, cruel and inexorable; she brings to our mind once more our weakness and helplessness Recent laboratory studies offer preliminary support for the Religious Comfort Model. Researchers have discovered that religious faith becomes more appealing after reminders of tragic suffering To our knowledge, no previous study has investigated whether the appeal of religion following a large-scale disaster is sufficient to affect rates of religious conversion. The timing of the Christchurch earthquakes between two NZAVS survey points enabled us to test the predictions of the Religious Comfort Model by comparing conversion rates in populations who were personally affected by the earthquake with populations who were not.those affected by the Christchurch earthquakes or living in the region of the earthquakes will be more likely to retain/convert to religion when compared with those who were not affected by the earthquakes or who were living in different regions.We emphasise that the Religious Comfort Hypothesis predicts that higher conversion rates will only occur among those affected by the Christchurch earthquakes. Comparisons with the rest of the New Zealand enabled us to better understand the specific contribution that the earthquake and its aftermath made to conversion rates.It has been proposed that in the wake of unusually stressful events, religious comforts offer practical benefits, however ideas about such benefits vary. Suggestions include that religious belief: (1) supports placebo healing Notably, many researchers remain skeptical about religious buffering. Karl Marx famously argued that religion persists as an empty distraction from the conditions of suffering Against dysfunctional theories, numerous studies have found that religion improves health, facilitates coping, It is important to note that the role religious faith plays in facilitating coping is often difficult to disentangle from other factors, such as community help and norms for healthy lifestyles. Religious denominations also vary in the support they provide for different stressors – Catholics, for example, find it harder to overcome divorce religious believers affected by the Christchurch earthquakes or living in the region will report superior health when compared with disbelievers.Here again, we used response data from those living outside of Christchurch who were not affected by the earthquakes to identify the specific effects of the natural disaster and aftermath, as compared to general processes that affected everyone in New Zealand, on subjective health.By world standards, New Zealand is a highly secular country, and it has grown increasingly secular during the past half century The Listener, accessed on 18 October 2012: http://www.listener.co.nz/current-affairs/christchurch-homeowners-face-long-wait-for-rebuild/].Though the overall benefits of religious and secular buffers might be similar, access to secular benefits might plausibly differ. At a philosophical level, religious orientations appear to provide all-embracing world-views with highly prescribed norms and practices Preliminary evidence for selective buffering comes from a recent study by Seirmarco and colleagues, who drew on retrospective reports about the importance of religion three years after the September 11 attacks those affected by the Christchurch earthquakes who lose religious faith (apostates) will experience health declines relative to believers and longer-standing secularists.The distinction between superior and selectively buffering requires a corresponding distinction between four categories of religious orientation:Believers: those who identified as religious both in 2009 and in 2011.Converts: those who were not-religious in 2009 and changed to a religious identification in 2011.Disbelievers: those who were neither religious in 2009 nor in 2011.Apostates: those who were religious in 2009 and changed to a not-religious identification in 2011.We compared changes in ratings of subject health for participants from each of these four categories by examining differences both across regions, and based on participant self-reports of being personally affected or unaffected by the earthquakes. These methods enabled us to distinguish earthquake-related interactions of subjective health and religious affiliation from general processes affecting the entire population.Proportional rates of change and stability in religious affiliation in the New Zealand population from 2009 to 2011 are presented in To determine rates of religious change among the earthquake-affected, we conducted parallel analyses using two measures of earthquake exposure: (a) by identifying participants by region of the earthquake (Canterbury) (b) by asking participants in the 2011 questionnaire to report whether they had been personally affected by the Christchurch earthquakes. Comparisons using regional data to examine change in religiosity among the New Zealand population affected by the Christchurch earthquakes are presented in However the data on religious conversion in the Canterbury region present a strikingly different pattern, revealing a net increase of 3.4% in religious faith. A chi-square test indicated the difference in the rate change by region was significant . As shown in We examined whether changes in religious affiliation for participants who were affected by the earthquake, relative to those who were not affected by the earthquake, were broadly consistent across different demographic groups, or whether they were more pronounced for specific demographic groups, such as men versus women, people of different ages, or different ethnic groups. We failed to detect any reliable differences in rates of apostasy and conversion moderated by these demographic factors, although religious conversion was 1.1% higher for women during the 2009 to 2011 period overall . Religious conversion rates were also 1.3% higher amongst minority groups relative to the New Zealand European majority, although again, this did not differ by region. These findings indicate that religious conversion may be more pronounced for women and ethnic minority groups, but these demographic differences seem to be independent of the effect of the Christchurch earthquakes.The religious conversion rate increased among those affected by the Christchurch earthquakes, but did having or acquiring a belief in God help? We focussed on subjective health evaluations, a key indicator of psychological well-being. To evaluate both superior and selective buffering effects we conducted a series of mixed-factor 2×2 ANOVA models comparing mean levels of subjective health evaluation in 2009 and again in 2011 (the repeated measures component) among those who were exposed or not exposed to the Christchurch earthquakes (the between persons component). We conducted these 2×2 models separately for believers, disbelievers, converts, and apostates. A significant interaction in any one of our ANOVA models would indicate that subjective health evaluations changed significantly over time, and that the extent of this change depended on exposure to the Christchurch earthquakes and their aftermath.The analysis focussing on self reports did not find a significant interaction between subjective health evaluations in 2009 and 2011 for religious believers  = .17, p = .766, partial Our analysis applying the same method (ANOVA models testing) to regional data gave a similar result. The interactions were non-significant for believers  = .02, p = .897, partial The Comfort Model of Religion has been the subject of longstanding philosophical debates. Several studies have shown that economic crises are associated with increases in religious practices Despite the interest of these findings, we offer a few words of caution about their interpretation. First, conversion is likely to be a complicated process Relatedly, “comfort,” as we have used this term, denotes a revealed preference for religion as quantified by conversion rates. The emotional mechanisms of religious commitment are complex Keeping these provisos in mind, it is remarkable that there was a significant overall increase in religious faith among the earthquake-affected. Consider: a third of the buildings in Christchurch were destroyed, including the Catholic and Anglican Cathedrals. Homes and businesses were lost. Many hundreds were badly injured. One hundred and eighty-five people lost their lives. Philosophers have plausibly argued that natural disasters such as the Christchurch earthquake are rationally incompatible with the existence of an all-powerful, all-loving God, because natural disasters cause pointless suffering to innocents, which a compassionate power would never permit Other studies have found that in the aftermath of natural disasters, religious faith helps with post-traumatic recovery How was the loss of faith related to the earthquake? Recall that Seirmarco and colleauges observed declines in the reported importance of religious beliefs several years after September 11 attacks were associated with reported severity of personal loss from the tragedy Setting aside causal ambiguities, it is important to remember that it does not take an earthquake to lose faith. Indeed, those who experienced the Christchurch earthquakes were, on the whole, more likely to identify as religious. That there was some apostasy in Christchurch, moreover, accords with a longstanding trend to apostasy in New Zealand It is important to recognise that our study leaves many questions unresolved. The association that we observed between loss of faith and diminished subjective well-being might be limited to New Zealand, or to similarly secular societies. Furthermore, the relationship between subjective well-being and biological health remains unknown. Perhaps appearances of stable health are linked to optimistic illusions Though many questions remain, we believe that our findings hold methodological, theoretical, and practical importance. Methodologically, our study reveals the power of within-country longitudinal designs to cast new light on fundamental questions of basic human concern. Within-country designs are complementary to transnational designs such as the World Values Survey, however within-country designs also offer distinctive advantages for addressing certain questions. Without access to regional data and personal reports, we simply could not have compared those who were affected by the earthquake with those who were not, and therefore, could not have evaluated the effects of the Christchurch earthquakes on religious faith and subjective health. Were we restricted to national averages, the signals of stability and change would have been obscured. We urge researchers in other countries to embark on national longitudinal projects, to better understand why core beliefs and values persist and change over time, and how their dynamics affect human flourishing.At a theoretical level, our study contributes to the old philosophical question of whether people seek religious faith at times of crisis (they do). Moreover, longitudinal data enabled us to more precisely investigate whether religion actually helps the faithful. The association between apostasy and diminished personal health suggests the need to distinguish between the benefits of having a religious faith and the costs of losing it. Secularisation has been associated with increases to security and income. In highly secular, affluent, and peaceful democracies such as New Zealand, we might expect a range of philosophical and institutional supports to protect individuals at times of crisis. Whereas religious converts might benefit from ready-made theologies and clearly identifiable communities of support, secular communities appear to be more diffuse, and less clearly associated with normative systems. Yet benefits from secular buffers might therefore be harder to access. Comparative longitudinal data afforded by the NZAVS revealed that it was a combination of the natural disaster and a loss of faith that was responsible for Christchurch apostates' declining sense of personal health. We hope the distinction between superior and selective buffering proves helpful to researchers investigating the costs and benefits of religious faith.Finally, our study is relevant to disaster relief. Sensible policies for addressing human misery rely on understanding the cultural and psychological resources available to those who need help. Though many questions remain unanswered, our results suggest that conversion to religious faith in the aftermath of a natural disaster is unlikely to improve subjective health, however de-conversion is associated with declines in subjective health. Whether religious or secular, those who want to foster healing in the aftermath of a devastating earthquake should consider joining arms in rebuilding the broken churches, for those who have faith.The NZAVS-2009 questionnaire was posted to 40,500 participants from the 2009 NZ electoral roll. The publicly available version of the roll contained 2,986,546 registered voters. This represented all citizens over 18 years of age who were eligible to vote regardless of whether they chose to vote, barring people who had their contact details removed due to specific case-by-case concerns about privacy. In sum, roughly 1.36% of all people registered to vote in New Zealand were contacted and invited to participate. The NZAVS-2009 sampled a total of 6,518 participants. The overall response rate  was 16.6%. These participants were contacted by postal mail and invited to complete a follow-up questionnaire one year later. The NZAVS-2011 contained responses from 3,914 of the same participants, a retention rate of 60.04% over the two-year period from 2009 to 2011. We limited our analyses to the 3,745 participants who completed the questions regarding their religious affiliation in both 2009 and 2011. The sample analysed here consisted of 2,305 women and 1,440 men with a mean age of 50.14 years in 2009 (SD = 14.94). The majority of participants  were NZ European/Pakeha. There were 14.2% (n = 532) identifying as Maori; 3.1% (n = 115) who identified with a Pacific Nation; 3.3% (n = 124) who identified as Asian; and 2.8% (n = 103) who identified as “other”. The education of participants reported in 2009 were as follows: 738 (19.7%) had no formal qualification, 1066 (28.5%) had a secondary school qualification, 606 (16.2%) had a tertiary diploma or trade certificate, 932 (24.9%) had an undergraduate university degree and 403 (10.8%) had a postgraduate qualification.Note that our sample was large . However, the initial response rate at Time 1 was fairly low (16.6%) and the retention rate from 2009 to 2011 was 60%. The low initial response rate was most likely because the study aimed to recruit people for a planned 20-year longitudinal investigation. The 2009 sample was reasonably representative of differences in the proportion of ethnic groups according to 2006 census figures. However, Pacific and Asian respondents were underrepresented in the 2011 wave: people who identified with these groups were slightly more likely to drop out of the sample. The NZAVS also oversampled women relative to men; however, as we noted earlier, differential changes across regions in religious affiliation were consistent when examining men and women separately. These caveats should nevertheless be kept in mind when generalising from our sample to the New Zealand population.We focussed on changes in religious affiliation depending on whether people were affected by the massive Christchurch earthquakes occurring in September 2010 and February 2011. Specifically, we examined change in religious affiliation both across regions and depending upon personal reports of whether participants were affected by the earthquakes. Our primary analyses focused on change in yes/no responses to the question “Do you identify with a religion and/or spiritual group?” asked in both late 2009 (before the earthquakes) and again to the same participants in late 2011. We thus identified four possible categories: Believers, Converts, Apostates, Disbelievers see . In bothWe measured exposure to the Christchurch earthquakes using two independent methods: (a) by identifying the region where participants were living in 2009 before the earthquakes; (b) by asking participants in the 2011 questionnaire to indicate whether or not they had been affected by the earthquakes. To measure self-reported earthquake exposure, the 2011 questionnaire asked, “were you personally affected by the Christchurch earthquakes?” allowing a yes/no response. We divided participant regions into two broad categories, those living in the Canterbury region in 2009 . Of the longitudinal sample that we analysed  a total of 501 (13.4%) respondents lived in the Canterbury region. This proportion is consistent with 2006 census figures indicating that 12.9% of the population lived in this region prior to the earthquake. In total, 801 (21.4%) individuals in the sample reported being personally affected by the Christchurch earthquake. Notably, our two measures of potential earthquake exposure overlapped: 370 respondents both lived in the Canterbury region and reporting being personally affected by the earthquake. These measures potentially index different aspects of earthquake exposure, however, as one relates to region and the other taps broader experiences that relate to being personally affected regardless of residential location. We thus report parallel results for all analyses using both of these indicators. Importantly, we find nearly identical results from both the affectedness report analysis and from the regional analysis.We indexed subjective evaluations of health by asking participants to rate their satisfaction with their level of personal health on a scale from 0 (completely dissatisfied) to 10 (completely satisfied). This item was drawn from the broader scale indexing different aspect of personal wellbeing developed by Cummings (2003) The data reported in this study were collected as part of a larger longitudinal research project, The New Zealand Attitudes and Values Study. The first phases of this longitudinal study were approved by The University of Auckland Human Participants Ethics Committee on 09-September-2009 for 3 years, reference number: 2009/336. Ethics approval for the study was re-approved by the University of Auckland Human Participants Ethics Committee on 17-February-2012 until 09-September-2015. Reference number: 6171.


In the modern era of neurosurgery, the use of the operative microscope, rigid rod-lens endoscope, and neuronavigation has helped to overcome some of the previous limitations of surgery due to poor lighting and anatomic localization available to the surgeon. Over the last thirty years, the supraorbital craniotomy and subfrontal approach through an eyebrow incision have been developed and refined to play a legitimate role in the armamentarium of the modern skull base neurosurgeon. With careful patient selection, the supraorbital “keyhole” approach offers a less invasive but still efficacious approach to a number of lesions along the subfrontal corridor. Well over 1000 cases have been reported in the literature utilizing this approach establishing its safety and efficacy. This paper discusses the nuances of this approach, including the benefits and limitations of its use described through our technique, review of the literature, and case illustration. Numerous neurosurgical approaches have been developed to operate on lesions of the frontotemporal skull base. These approaches include frontal, bifrontal, frontotemporal, pterional, orbitozygomatic, and other variations . The evoThe supraorbital craniotomy and subfrontal approach have been used to access a number of pathologies including tumors  and vascular abnormalities  , 2, 5–35When considering any approach to a pathological entity, it is important to understand the advantages and disadvantages of a given procedure. Surgery through an eyebrow incision may not be appropriate for all lesions of the anterior skull base. There is a narrow viewing angle through this approach that may require frequent adjustment of the operating room table and microscope for adequate visualization of a given lesion. The microscope light is often another problem, as there may be some difficulty getting adequate light through such a small opening onto a deep-seated lesion. Microinstruments require almost coaxial control through such narrow anatomic windows , 5. In tUse of a rigid rod-lens endoscope in combination with the operative microscope can provide a great benefit with the supraorbital craniotomy and subfrontal approach. The endoscope can provide a much greater light source at the depths of the exposure, with greater focus and better visualization. Ensuring a large enough size to the craniotomy  is important as well to ensure adequate maneuverability of instruments for a bimanual approach to surgery [After general anesthesia, endotracheal intubation, and placement of a Foley catheter, the patient is fixed in a Mayfield three-pin head holder with two pins on the ipsilateral posterior cranium and the one pin site on the contralateral frontal bone. The torso is slightly elevated at ten degrees, and the head is positioned in a slightly extended position of around 15–20 degrees to allow gravity retraction of the frontal lobes away from the surgical field. No retractors are used. The head is turned approximately 15–45 degrees contralaterally to the side of surgery to allow appropriate visualization of midline lesions. The bed can be further rotated as necessary for further adjustments during surgery. Midline lesions, such as olfactory groove lesions, require more rotation, whereas laterally placed lesions require less rotation for appropriate visualization and access. The most important information in decision making regarding the side of the approach is the structure of the lesion itself and its relationship to surrounding anatomic structures. Certainly, when either side can adequately access the lesion, we typically choose a nondominant approach in order to reduce the risk of damage to the dominant frontal lobe.The skin incision is made along the eyebrow without cutting the hair of the eyebrow . PreviouThe craniotomy is made by bluntly dissecting a small portion of temporalis muscle and fascia at the superior temporal line and drilling a 5 mm burr hole on the lateral aspect of the exposure below the temporalis for a better cosmetic result. Care is taken to avoid the use of cautery around the temporalis at this location, as this may cause damage to the frontalis branch of the facial nerve. A craniotome is then used to make two cuts. The first is from the burr hole along the floor of the anterior cranial fossa extending to a position lateral to the supraorbital notch. The second again starts from the lateral burr hole but makes an arch superiorly to then return to meet the medial edge of the first cut. The craniotomy takes the form of a “D,” with the back wall of the “D” along the floor of the anterior cranial fossa. It is important to ensure a craniotomy at least 1.5–2 cm in width, or manipulation of microinstruments is very difficult. It is also important to recognize a breach of the frontal sinus, as this can be a source of CSF leak postprocedure if not adequately addressed. In fact, a very lateral extension of frontal sinus may preclude the use of this approach in a given patient because of the difficulty repairing a large opening in the frontal sinus via this approach. We have used bone wax to seal off any small breach of the frontal sinus and betadine-soaked gel foam to seal off larger defects.The dura is now dissected off the orbital roof. At this point, the inner table of the inferior edge of the craniotomy is drilled flush with the orbital roof. Any ridges of the orbital roof can also be leveled with the high-speed drill. This not only improves visualization but also allows greater access of instruments during the procedure. A malleable brain retractor may be placed against the dura to protect against unintentional durotomy. The outer table is left intact to maintain cosmesis. Bone dust is washed out with antibiotic irrigation prior to dural opening. The dura is opened in a “C”-shaped fashion and reflected inferiorly with a stitch. The microscope is brought into the field, the frontal lobe is lightly retracted with a cottonoid, and the CSF cisterns are opened to allow CSF egress to facilitate brain relaxation. Following brain relaxation, the primary procedure may be performed safely with no fixed retractors on the brain and with use of the operative microscope, a rigid rod-lens endoscope, or both. Wound closure is straightforward. The dural leaflets are reapproximated with a 4-0 Nurolon suture sewn in a running fashion. The craniotomy bone flap is replaced with a titanium burr hole cover and two titanium square plates to improve the cosmetic result by restoring the supraorbital ridge. The pericranium and muscle flap are then closed primarily. Buried, interrupted, and absorbable sutures are used in the dermis, and a 5-0 prolene subcuticular stitch is placed without any knots to ensure removal in the office in 7–10 days. A head wrap can be applied until the first postoperative day to lessen subgaleal edema formation.A number of case series utilizing this approach have been published in the literature . The repA 71-year-old RH woman presents with a history of progressive headaches who underwent an MRI of the brain with gadolinium contrast administration. The MRI demonstrated a homogeneously enhancing sellar/suprasellar lesion that extended to the planum sphenoidale causing optic chiasmal compression as well as compression of the right optic nerve. The right A2 branch of the anterior cerebral artery coursed through the superior aspect of the tumor. Its imaging characteristics were most consistent with a tuberculum sellae meningioma. This increased in size on subsequent imaging, and the patient underwent elective resection of her tumor by a right supraorbital keyhole craniotomy through the right eyebrow. Preoperative and postoperative imaging are shown . She hadA 46-year-old right-handed male presented with visual loss and headache. He could only finger count in the right eye and had an additional bitemporal hemianopsia on visual field testing. Imaging demonstrated a homogeneously enhancing mass involving the tuberculum sellae and planum sphenoidale causing compression of the right optic nerve and chiasm. Imaging characteristics were most consistent with a meningioma. He underwent a right supraorbital craniotomy via an eyebrow incision obtaining a Simpson grade II resection . PostopeA 51-year-old right-handed woman presented with vision loss and headache. Imaging demonstrated a sellar and suprasellar heterogeneously enhancing cystic mass causing optic chiasmal compression . She undWhen the supraorbital craniotomy and subfrontal approach through an eyebrow incision were first described, there was significant controversy over the use of this approach in neurosurgery , 5. ManyExperience has helped to demonstrate the limitations of the approach, and many of these early limitations have been overcome. A number of case series reported in the literature demonstrate the efficacy of this approach see . Gross tEntering through the eyebrow historically led to postoperative loss of supraorbital sensation or to a palsy of the frontalis branch of the facial nerve see . PlacemeIn the setting of vascular pathologies, there may be some difficulty with using two suction tubes in managing prematurely ruptured aneurysms or to obtain proximal control , 46, 47.Numerous shortcomings have been overcome since the introduction of this approach in the 1980s. Probably the biggest limitation was the problem of lighting with the operating microscope down such a narrow corridor. Endoscopes have dramatically improved visualization of this region through this approach and allow for safer dissection with better visualization through this smaller incision than can often be achieved with the microscope alone. Endoscopic-assisted surgery is a common adjunct to the modern skull-based surgeon wishing to employ this keyhole approach in his armamentarium, and is discussed in more detail in what follows.Proper positioning of the head for the keyhole supraorbital craniotomy can play an important role in surgical access of skull base lesions. Extension of the neck permits frontal lobe relaxation in combination with mannitol. Contralateral rotation of the head is also performed , 13, 48.Multiple cadaveric studies have been performed in an attempt to increase the safety of the supraorbital keyhole approach. One study looked at the location and course of the supraorbital nerve and the frontalis branch of facial nerve. This study of ten specimens noted a supraorbital notch in 12/20 sides (right or left) and a supraorbital foramen in the remaining 8 . The latAn additional cadaveric study sought to quantitatively verify the accuracy of the claims of Perneczky's group that the optical field widened with increasing distance from the keyhole and that contralateral parasellar structures could be visualized well –5, 46. IEndoscopes have aided in overcoming one of the main disadvantages to the keyhole approach: illumination. Use of the microscope in keyhole surgery requires frequent changing of the visual angle to allow illumination of the area of interest deep in the surgical field. Endoscopes produce illumination at depth rather than from a distance and therefore can illuminate the area of interest without casting shadows on the field. Endoscopes can be held either by an assistant or with a retractor arm, allowing the surgeon to continue to work bimanually with microinstruments running in a parallel axis with the endoscope . Angled A few case series have been reported regarding both supraorbital keyhole approach or endoscopic endonasal extended approaches for resection of tuberculum sellae meningiomas. One author performed a meta-analysis comparing the endoscopic endonasal extended approach for tuberculum sellae meningioma resection with an open craniotomy approach . In thisThe supraorbital keyhole approach has also been described for resection of olfactory groove meningiomas. In the literature, a MEDLINE search revealed a total of 81 cases reported in the literature where outcomes data were specific to the olfactory groove location of the tumor –22, 34. A case series of 43 patients was recently reported with either craniopharyngiomas or anterior skull base meningiomas resected through either an extended endoscopic endonasal route or through a keyhole supraorbital craniotomy and subfrontal approach . Of the The location of the chiasm in relation to the tumor, along with the lateral extension of tumor, may determine whether a supraorbital keyhole or endoscopic endonasal approach is taken. Prechiasmatic craniopharyngiomas may be better accessed through a supraorbital keyhole approach especially if there is lateral or suprachiasmatic extension of tumor. Retrochiasmatic lesions, on the other hand, can pose a greater chance for injury to the visual apparatus through a supraorbital approach and may be better resected through an endoscopic endonasal approach . Cosmesis has prevented many surgeons from attempting this approach or has led to their abandonment of this approach with its introduction early on. A number of modifications have led to what many now consider to be a superb cosmetic result with the supraorbital craniotomy and keyhole approach. A limited skin incision within the eyebrow, minimal temporalis muscle dissection, a small bone flap, and closure with the orbicularis oculi muscle/pericranium layers have contributed to the success of the eyebrow incision. Temporalis muscle atrophy, so common with standard frontotemporal and pterional craniotomies, can be avoided with the eyebrow incision . Of courThere have been a number of ways to perform the incision including superciliary, transciliary, and even transpalpebral incisions in an attempt to improve cosmesis , 26, 29.Another important cosmetic consideration is performing the initial incision through the skin and dermis layers only. Cephalad dissection superficial to the orbicularis oculi, pericranium, and temporalis muscle is important for development of a separate tissue flap for covering the keyhole craniotomy during closure , 22, 46.The supraorbital craniotomy and keyhole approach through the eyebrow permit access to a number of lesions in the subfrontal corridor with minimal brain retraction and a much smaller area of potential injury of superficial structures. All minimally invasive techniques have a learning curve, and smaller, simpler lesions should be performed first through this approach before moving on to larger, more complicated lesions. Our experience is that midline and suprasellar lesions are more easily accessed through this approach than laterally based lesions. Endoscopy can play an important role in improving visualization through the keyhole corridor. Attention to detail can allow this approach to be performed with superb cosmetic results while still achieving surgical efficacy and limiting complications.


Analyses based on individual growth curve modeling showed that participants in the experimental schools displayed better positive youth development than did participants in the control schools in terms of different indicators derived from the Chinese Positive Youth Development Scale, including moral competence and behavioral competence and cognitive behavioral competencies. Significant results were also found when examining the trajectories of psychological development among control and experimental participants who perceived the program to be beneficial. Findings based on longitudinal objective outcome evaluation strongly suggest that the Project P.A.T.H.S. is effective in promoting positive development in Hong Kong secondary school students.To assess the effectiveness of the Tier 1 Program of the Project P.A.T.H.S., a randomized group trial with eight waves of data collected was carried out. At the fifth year of data collection, 19 experimental schools ( Adolescence is an age of transition. With physical and cognitive maturation taking place in puberty, intrapersonal, and interpersonal changes in adolescents intensify and there are growing social demands and expectations during adolescence. Because of these changes, adolescence is also regarded as an age of stress. The stressors confronting an adolescent might include family stressors , interpersonal stressors , academic stress , living circumstances-related stressors , financial stressors , developmental stressors , psychological stressors , and social stressors . In a Chinese society such as Hong Kong, stress arising from academic excellence and subtle social competition is particularly relevant for Chinese adolescents. Obviously, how to cope with stress in adolescence is an important developmental task for adolescents. At the same time, there are several myths about development of adolescents in the Chinese culture . These ihttp://www.casel.org/). Generally speaking, several SEL attributes are commonly included in different SEL models. These include self-awareness (identifying emotions and recognizing strengths), social awareness (perspective-taking and appreciating diversity), self-management , responsible decision making , and relationship skills . Sun and Shek [ One obvious way to nurture young people is to promote social and emotional competencies of young people . Accordiand Shek  showed t How is SEL implemented in Hong Kong? Instead of focusing on life skills and competencies in students, moral education focusing on values has been emphasized in Hong Kong. While the policy and curriculum guide governing moral education are elegant, there are three problems in the related policy and its actual implementation. First, there are no curriculum materials which have been validated, although there is a pool of suggested curriculum materials that can be used by teachers. Second, there is a wide variation in the mode of implementation of moral education in schools settings. While some schools may incorporate moral education in the formal curriculum, some may use extracurricular activities such as morning assemblies to implement moral education. Finally, rigorous evaluation of moral education programs is rare, although there are administrative audits and collection of management information by the government. The lack of rigorous evaluation implies that there is no way to understand whether there are changes in outcomes as such moral values of the students because input and output evaluation is simply not adequate. Shek and Yu  reviewedThe Project P.A.T.H.S.  is a youth enhancement program that attempts to promote holistic youth development in Hong Kong. While the Tier 1 Program is a universal positive youth development program adopting a curricular-based approach for students in Secondary 1 to Secondary 3, the Tier 2 program is designed for students with greater psychosocial needs. As far as objective outcome evaluation is concerned, several studies have showed that students who participated in the project showed better development than those who did not participate. Based on the first two waves collected in a randomized group trial, Shek et al.  showed tWith reference to the first six waves of data in the junior secondary school years , evaluation findings showed that the Project P.A.T.H.S. was able to promote holistic development in the participants. Individual growth curve analyses showed that participants in the experimental schools displayed better positive youth development than did participants in the control schools based on different indicators derived from the Chinese Positive Youth Development Scale, including positive self-identity, prosocial behavior, and general positive youth development attributes. Differences between experimental and control participants were also found when students joining the Tier 1 Program and perceiving the program to be beneficial were employed as participants of the experimental schools . Similar To replicate the objective outcome evaluation findings and to examine the long-term effectiveness of the Project P.A.T.H.S. over a period of five years, the Wave 8 data were included in the present study. Essentially, we asked whether the program effect could be sustained over a period of two years after the termination of the project at Secondary 3. In the realm of science, replication plays an important role. As pointed out by Campbell and Stanley , “we musDuring 2006–2011, a total of 7,846 Secondary 1 students  were recruited from 48 schools. Shek and associates , 13 descStudents were measured at baseline in the fall of 2006 (Wave 1) and then followed longitudinally across waves . In Year 1 (2006-2007), one school withdrew after Wave 1. In Year 2 (2007-2008), Waves 3 and 4 data were collected from the same cohort, with 20 experimental schools  and 24 control schools. In Year 3 (2008-2009), Waves 5 and 6 data were collected from the same cohort with 19 experimental schools  and 24 control schools. A total of 3,820 students completed all 8 waves of the study (49%). In the present study, all data were tested as individual growth curve model allows unequal interval spaced time points and missing data . The numAt pre- and posttest, the purpose of the study was mentioned, and confidentiality of the collected data was repeatedly emphasized to all students in attendance on the day of testing. Parental and student consent had been obtained prior to data collection. All participants responded to all scales in the questionnaire in a self-administration format. Adequate time was provided for the participants to complete the questionnaire. A trained research assistant was present throughout the administration process.Consistent with the procedures used in Year 1, the participants were invited to respond to a questionnaire that comprised different measures of youth development at pretest  and posttest . The following measures were used.Based on the analyses conducted in Year 1, the item composition of the 15 subscales of the CPYDS is as follows.Bonding Subscale (six items).Resilience Subscale (six items).Social Competence Subscale (seven items).Emotional Competence Subscale (six items).Cognitive Competence Subscale (six items).Behavioral Competence Subscale (modified five items).Moral Competence Subscale (six items).Self-Determination Subscale (five items).Self-Efficacy Subscale (modified two items).Beliefs in the Future Subscale (modified three items).Clear and Positive Identity Subscale (seven items).Spirituality Subscale (seven items).Prosocial Involvement Subscale (five items).Prosocial Norms Subscale (five items).Recognition for Positive Behavior Subscale (four items).As mentioned by Shek , differeIndividual growth curve (IGC) is an advanced statistical technique which is conducted to examine “aggregates” of individual curves rather than separate analysis of each individual growth curve . This meIGC is an appropriate approach in studying individual change as it creates a two-level hierarchical model that nested time within individual –20. The Level 1 model:β0 is the initial status  of the outcome variable for individual i. β1 is the linear rate of change for individual i and eij is the residual in the outcome variable for individual i at Time t. Yij is the repeatedly measured of the outcome variable for an individual i at Time t.In our study, Time  remains, while 2Time  and 3Time  are added in the model:To test a nonlinear individual growth trajectory across time, other higher-order polynomial trends  can also be included for model testing. This is showed in , in whicgroup in the present study) is included to analyze the predictor's effect on interindividual variation of outcome variable. The errors are assumed to be independent and normally distributed and that the variance is equal across individuals [The Level 2 model captures whether the rate of change vary across individuals in a systematic way. The growth parameters  of Level 1 are the outcome variables to be predicted by the between-subjects variables at Level 2. At this level (In ), an expividuals .The Level 2 model isYij is the grand mean for the outcome variable for the whole sample at Time t. γi0 is the initial status of the outcome variable for the whole sample at Time t. γi1 is the linear slope of change relating to the outcome variable for the whole sample at Time t. γi2 is the quadratic slope of change relating to the outcome variable for the whole sample at Time t. γi3 is the cubic slope of change relating to the outcome variable for the whole sample at Time t. γ01, γ11, γ21, γ31, are used to test whether the predictor  is associated with the initial status, linear growth, quadratic growth, and cubic growth, respectively. roi, ri1, and εij are the residual errors that is not explained by Level 2 predictors.In our study, In this study, we tested whether treatment was predictive of students' growth parameters  in several positive youth development indicators across time. In particular, the relationships between these indicators and group were estimated after controlling the effect of gender and initial age. The intercept  and linear slope were allowed to vary across individuals. To examine the amount of total variation in the outcome variables that is related to between-individual differences, the intraclass correlation coefficient (ICC) is calculated.group—control versus experimental groups) as a predictor. Participants in the control group were coded as −1 and those in the experimental group as 1. Two covariates  were included when examining the predictive program effect on the outcome variables. Gender was coded as −1 = male and 1 = female. Similar coding method for a dichotomous variable was found in previous studies [Initial age was then centered by subtracting the mean age, and therefore, the centered initial age was generated.A dummy variable was created . Control group dropped slower , but decelerated faster than did the experimental group  across 8 waves. Similar trend that was also found in BC, except the test of group differences in linear slope was not significant . These results revealed that both groups differed in their rates of growth over time and these differences occurred up through Wave 7 after which they diminished gradually  in Hong Kong by using a validated measure of positive youth development—the Chinese Positive Youth Development Scale. This is the first known scientific study that adopted a randomized group trial design using longitudinal data to evaluate a positive youth development program in the Chinese context. Consistent with previous longitudinal results –7, 31, pRecent analyses based on this cohort revealed that the positive impact of the Project P.A.T.H.S. on reducing problem and risk behaviors . Given tIn the present study, significant results that were more pronounced among those who perceived the program to be effective deserves some discussion. First, the findings suggest that positive youth development programs might not work in a “stimulus-response” manner and the cognitive appraisal of the participants is in fact important in determining the program outcome. As this factor is not properly addressed in the literature, it is recommended that further study should be carried out to examine how the subjective appraisal of the program participants might affect program outcomes through alteration of their motivation to join and participate. Second, the findings suggest that it is important to attend to the subjective outcomes perceived by the participants. As such, how to promote a sense of success and program ownership may help to promote program effectiveness in the long run.Another significance of the study is the use of individual growth curve modeling to evaluate the impact of a positive youth program in a large sample across a five-year period, which is scarce in positive youth program literature. Researchers noted the needs of using advanced modeling techniques in longitudinal research , 38, 39.There are several limitations in the present study. First, the data were based on self-report measures. Future research should evaluate the program by collecting longitudinal information from multiple methods  and sources . Second, future research should examine the longitudinal effect of the positive youth development qualities on promoting psychological well-being  and reducing problem behaviors . This is supported by recent longitudinal findings which showed that positive youth development programs such as the Project P.A.T.H.S. can help to promote youth development and reduce their negative behavior among Hong Kong adolescents –10. Desp


Policy-makers who are making decisions on sexuality education programs face important economic questions: what are the costs of developing sexuality education programs; and what are the costs of implementing and scaling them up? This study responds to these questions by assessing the costs of six school-based sexuality education programs .Cost analyses were carried out in schools that were fully implementing a SE program, as this best reflects the resources needed to run an effective program. The costs were analyzed from the program perspective, meaning that all costs borne by the governmental and  non-governmental organizations supporting the program were included. Cost analyses were based on financial records, interviews and school surveys.We distinguished costs in three consecutive program phases: development, update and implementation. Recommendations on the most efficient program characteristics and scale-up pathways were drawn from results of three fully scaled up programs , scale-up scenarios of two pilot programs (Kenya and Indonesia), and an implementation plan (India), The costs of the programs were compared by converting cost per student reached in US dollars (US$) to international dollars (I$).Findings revealed a range of costs and coverage of sexuality education programs. Costs per student reached were; US$7 in Nigeria, US$13.50 in India, US$33 in Estonia and the Netherlands, US$50 in Kenya, and US$160 in Indonesia.Intra-curricular sexuality education programs have, because of their compulsory nature, the most potential to be scaled up and are therefore most efficient. Extra-curricular sexuality education programs have lower potential to be scaled up and are therefore less efficient. In terms of class size and number of lessons, countries need to strike a balance between the quality  and the costs (demanding larger classes and fewer lessons). Advocacy was a significant cost component. Policy-makers worldwide who are involved in decisions about sexuality education programs face important economic questions: what are the costs of developing sexuality education programs; and what are the costs of implementing and scaling them up? Knowing the answers to these questions would enable policy-makers to invest resources in sexuality education more efficiently. This study responds to the above questions by assessing the costs of school-based sexuality education (SBSE) programs in six countries  and program scale-up projections in two countries (Kenya and Indonesia).This study fills in important gaps on the economic aspects of sexuality education programs worldwide, in low-, middle- and high-income countries. It also comes at a time when interest in sexuality education programs is growing considerably.This article only considers costs of SBSE programs. Health effects of these SBSE programs are notoriously difficult to measure. As an integrated part of our study we made efforts to measure health effects of the Estonian program, which will be reported in a companion paper .The cost estimates in this article are relevant not only to the countries and sexuality education programs studied, but also to other countries considering implementing or scaling up existing sexuality (including HIV prevention) education programs.The selection of countries reflects a broad geographical spread, with two countries in Africa (Nigeria and Kenya), two in Asia (Indonesia and India), and two in Europe (Estonia and the Netherlands). It also reflects a range of experiences. The Netherlands has a fairly long tradition of sexuality education, whereas in Indonesia and Kenya, recently implemented pilot programs are evaluated, operating on a relatively small scale. Estonia introduced its program about 15 years ago, though it is only now firmly established nationwide. In Nigeria, the sexuality education program started in Lagos state, and is now also implemented in Abuja state. India is in the course of implementing a program in the Orissa State. Table The success of sexuality education programs is largely determined by the context in which they are developed and implemented, as well as by their characteristics and the quality of implementation. In many countries, sexuality, and therefore sexuality education, is a sensitive issue that may generate opposition. Where there is opposition, the introduction of sexuality education requires careful planning and a wide variety of advocacy and public education activities. This has a significant effect on the costs and impact of the programs (see below).Among international experts there is a strong consensus that sexuality education programs that are fully integrated into the school curriculum are preferable to stand-alone programs. However, in many countries, the conditions for fully integrated sexuality education programs are not sufficient, and therefore extra-curricular, stand-alone programs are the only ones that may currently be possible. The programs in Indonesia and Kenya are of the latter type.plans. More detailed costing methods and resource use and prices can be found elsewhere [This report presents the results of a comprehensive costing analysis – based on detailed inspection of financial records, interviews with sexuality education program personnel, and primary data collection through specifically designed surveys in schools – in all countries studied. Cost analyses were carried out in schools that were fully implementing a SE program, as this best reflects the resources needed to run an effective program. The exception is Orissa State, India, where program implementation began in 2010, and where the costing analysis is therefore estimated on the basis of implementation Analyses were conducted from the program perspective, including all costs as borne by governmental and  non-governmental organizations supporting the program. The economic costs of the sexuality education programs were estimated, including all resources used, and the budgetary outlays were also calculated, i.e. the costs of running the programs additional to already existing expenses for teacher salaries. Various program phases – development or adaptation, implementation , and update – and attendant inputs were identified to reflect all resources required for developing and implementing a sexuality education program. Recommendations on the most efficient program characteristics and scale-up pathways were drawn from results of three fully scaled up programs , scale-up scenarios of two pilot programs (Kenya and Indonesia), and an implementation plan (India).All costs were analyzed in local currencies and converted to US dollars (US$) using 2009 exchange rates . The iniInternational dollars have the advantage that they account for the difference in price levels between countries, and allow for a comparison of the actual resource use by the sexuality education programs in the countries concerned.The overall approach adhered to the WHO-CHOICE methodology on costing analysis – an internationally accepted standard for the conduct of economic analysis of health programs, especially in low- and middle-income countries .Study findings reveal a wide range of costs and coverage of sexuality education programs across the countries studied. Total costs of sexuality education programs, including development or adaptation, updating and implementation, range between US$1.19 million in Indonesia to US$12.1 million in the Netherlands. The total number of students reached varies from some 6,000 in Indonesia, to 990,000 in India (as planned for the period 2010–2014). This is dependent on the scale of the program and the number of years it is implemented in the country, and therefore the report concentrates on annual figures. The annualized costs and the annual number of students reached in 2009 are US$562,000 and 246,000 students in Nigeria; US$364,000 and 7,300 students in Kenya; US$289,000 and 1,800 students in Indonesia; US$3.5 million and 780,000 students in India (as planned in 2014); US$311,000 and 28,000 students in Estonia; and US$830,000 and 25,300 students in the Netherlands. In every country, the majority of all costs are implementation  costs, and costs of program development, adaptation and updating are minor. Table In the interpretation of the main findings – the cost per student reached across countries – it must be stressed from the outset that the sexuality education programs in Kenya and Indonesia are much more costly because they are still in a pilot phase and therefore small scale. Costs per student reached were US$7 in Nigeria and US$13.50 in India, US$33 in Estonia and the Netherlands, US$50 in Kenya, and US$160 in Indonesia.Figure However, if only budgetary outlays are considered, i.e. the costs in addition to regular expenses on teacher salaries, costs per student reached fall to US$0.60 in Nigeria, US$2.50 in India, US$8 in Estonia, US$10 in the Netherlands, US$37 in Kenya and US$135 in Indonesia. In India, Estonia and the Netherlands, for example, these budgetary outlays constitute 0.5, 0.2 and 0.1 per cent, respectively, of current expenditure per student in secondary education.Results of the scale-up scenarios of Kenyan and Indonesian programs suggest that the most efficient strategy is to first increase program uptake in the current schools, for example by making the curriculum mandatory, before introducing the program to new schools or districts. Moreover, in case these pilot programs would be scaled up to 300 schools and achieved (mandatory) full coverage, costs per learner reached would be reduced to US$16 in Kenya and US$13.40 in Indonesia.The sexuality education programs in Nigeria, Kenya, Indonesia and India have been implemented in contexts where sexuality, and therefore sexuality education, is a sensitive issue. In contrast, sexuality education is not a sensitive issue in Estonia or the Netherlands. The sensitivity of the topic has important consequences for how and the pace at which sexuality education programs can be introduced, their character (comprehensive versus abstinence-only), and the scale at which they can be carried out. This has an effect on costs and potential impact. In Nigeria and India, sexuality education programs initially came to a halt because of socio-cultural opposition, thereby causing years of delay and related loss of investments. In Nigeria, the initial comprehensive program had to be reduced: all elements related to actual sexual and preventive behavior, including contraception and condoms were removed. The programs in Orissa State, India (as planned) and Estonia are good examples of comprehensive, integrated and fully scaled up sexuality education programs, and these hold important lessons for other countries that wish to achieve similar scales and related impact. The programs in Kenya and Indonesia are NGO-initiated, also in response to the sensitivity of sexuality education and the relative hesitance of national governments to address the topic. These programs are extra-curricular, voluntary and seem to be constrained in the coverage they can achieve. However, they can be an important stepping stone toward the development of national sexuality education programs. All sexuality education programs in all countries require careful planning and a wide variety of advocacy and public education activities to achieve their implementation.The programs in Nigeria, India, Estonia and the Netherlands appear to be relatively inexpensive in terms of cost per student reached, costing approximately US$7, US$14, US$33 and US$33, respectively. These programs are all intra-curricular and implemented on a large scale , which reduces costs per student of national and state-level activities, such as program development, management and advocacy. Also important is the mandatory student enrolment in these programs, resulting in an almost comprehensive coverage of enrolled students per school. This reduces school-level costs per student, such as teachers’ salaries . On this basis, we conclude that intra-curricular sexuality education programs are most efficient, and we refer to the program in Estonia and the planned program in India as best examples in this respect. The sexuality education program in the Netherlands is difficult to interpret in this context, because the program is relatively short, and functions as an addition to a more elaborate sexuality education at primary school, and in biology lessons in secondary school.The sexuality education programs in Kenya and Indonesia appear to be relatively costly, at US$50 and US$160 per student reached, respectively. These programs are currently in the pilot phase, geographically spread out, and initiated by international and national NGOs. At this stage, these programs have limited coverage – annually reaching between 1,800 and 7,300 students respectively – and carry high operations  costs. Cost per student would diminish considerably if the programs were scaled up beyond the pilot phase. However, both programs are also extra-curricular and thus voluntary, so the potential of such programs to achieve widespread coverage is questionable. Integration of the program in the regular curriculum would be a possible strategy to meet that concern. However, such programs may sometimes be the only available option in a country where sexuality education is a sensitive issue, and this could be a reason to accept their relatively high cost during a period of transition.In addition, the sexuality education programs in Kenya and Indonesia are both computer-based, and this also makes them relatively costly because it necessarily reduces class size (schools have a limited number of computers). Uptake in schools – between 42 students per class in Indonesia and 44 students in Kenya – is constrained as a result. This hinders the scale-up of a program across and within schools, and its integration into the regular curriculum.Teacher salaries are a major cost component in all programs, and class size strongly influences cost per student reached. In Nigeria, classes for the sexuality education lessons usually have 75 to 150 students, while classes are much smaller in, for example, India (around 40 students) and Estonia (around 18 students). While large classes are thus favorable for cost purposes, the quality of implementation in such classes will likely be compromised. Even when specific strategies are developed in sexuality education to cope with large classes, as in Nigeria, sexuality education typically requires interactive teaching methods with high levels of student involvement, which can hardly be realized in overcrowded classes.Advocacy costs are a significant cost component in all countries, ranging between 4 per cent of total costs in Kenya to 13 per cent of total costs in Indonesia. The only exception was the Netherlands, where advocacy costs were 0.1 per cent. Therefore, advocacy costs seem to be highest where there is most resistance towards sexuality education. Advocacy costs are incurred not only in the development phase of the program but also throughout its implementation, and reflect the sensitive nature of sexuality education curricula in a country. Advocacy includes a broad scope of activities including political lobbying, media activities, stakeholder meetings, working groups, sensitization meetings for school staff, parents, and health care providers, and exhibitions.Programs examined in this study differ in the way they were developed and/or adapted. Development costs in Estonia were low because of low salary levels during the first years of independence, and programs did not require intensive advocacy activities. Programs in Kenya and Indonesia were adapted from a similar program in Uganda and had important savings in the development costs of the original software. However, the adaptation process was still relatively costly as these computer-based programs required expensive software adaptation activities. Moreover, the adaptation was supported by an international organization, which added extra costs. The adaptation costs in these countries constituted between 15 per cent and 24 per cent of total costs. Development and updating costs ranged between 1 per cent and 11 per cent of total costs in the other study countries. Finally, the proportion of development or adaptation costs is also dependent on the number of years a program has been implemented in a country.The duration of the programs varies. The number of learning hours per student over the duration of the curriculum varies between 11 hours in the Netherlands , to some 40 hours in most other countries. Obviously, this is closely related to the cost per student reached. The number of learning hours also determines the impact of a program – international standards recommend at least 12 to 20 lessons  over several years.The budget impact of implementing sexuality education programs is not equal to the economic costs as presented in this study. Teacher salaries are included as economic costs in this study but are a regular expense of the Ministry of Education, irrespective of the implementation of sexuality education programs. In the implementation of a sexuality education program, these salaries would therefore not incur additional budgetary outlays. This study shows that budgetary outlays are less than 25 per cent of the economic costs of sexuality education programs (with the exception of Kenya and Indonesia) and, as estimated in this study, range from US$0.60 in Nigeria to US$10 in the Netherlands. However, it should be noted that the introduction of a sexuality education program in the curriculum could come at the expense of not teaching other courses, which is an opportunity cost.In order to assess the cost implications of scaling up sexuality education programs, we defined several different scenarios and, based on these, we recommend the most efficient pathways to greater sexuality education coverage. The most efficient strategy appears to be to start first expanding program uptake in schools where the program is currently being implemented, for example, by making the curriculum mandatory, before introducing the program to new schools or districts. Again, this is because teacher salaries constitute a major cost component and are reduced by covering more students per class or school. The ideal strategy from the efficiency point of view is to maximize uptake in schools and the coverage of schools in a country.A number of limitations were faced in conducting the study. First, information on the actual number of learners reached was not always available, especially in the larger intra-curricular programs. In those instances, program coverage was calculated on the basis of secondary sources, e.g. student materials used or teachers trained. Second, it is not always clear where sexuality education programs begin and end. Sexuality education programs are sometimes part of wider life-skills programs, as in Estonia. Since general life skills, such as decision-making competence, serve more purposes than only promoting healthy sexual behavior, it is then somewhat arbitrary to determine which part of the program should be labeled sexuality education. Third, it was not always possible to make detailed assessments of costs. Some programs, such as curricula in Estonia and the Netherlands, have existed for a long time, and financial records were not always available. In those instances, the analysis was based on gross estimates from program personnel.This study only focuses on the costs of sexuality education programs. It does not answer several other very important questions, such as variation in the quality of different types of programs, nor does it address the important question of how to develop a sexuality education program and integrate it into existing curricula. This requires additional efforts beyond the scope of this study, which could result in a strategic document that outlines pathways, under different conditions, for successfully developing and integrating sexuality education in school curricula.On the basis of the above analysis we draw the following conclusions and recommendations.Intra-curricular sexuality education programs have, because of their compulsory nature, the most potential to be scaled up – in terms of coverage of schools and students in schools – and are therefore most efficient. Where possible, we recommend this type of sexuality education program.Extra-curricular sexuality education programs have, because of their voluntary nature, lower potential to be scaled up and are therefore less efficient. These programs are therefore not recommended. However, they can be important learning experiences and stepping stones to national sexuality education programs, or may be the only available option in a country considering the sensitive nature of sexuality education. Where possible, such programs should be gradually integrated in the national curriculum to render them more efficient.In terms of class size and number of lessons, countries need to strike a balance between the quality  and the costs (demanding relatively large class sizes) of sexuality education programs. Furthermore advocacy appeared to be a significant component of the costs of sexuality education programs in most countries, and we recommend that educational authorities consider this to be a necessary investment. Next we recommend that new initiatives save costs by adapting existing programs to their own  context, instead of developing new ones. We also recommend that sexuality education programs wishing to increase their coverage start by expanding program uptake among students in schools first (e.g. by making the curriculum mandatory), before introducing the program to new schools or districts. Finally where student access to computers is limited, computer-based sexuality education programs are not recommended.The authors declare that they have no competing interests.JK is a health economist consultant. EK is a sexual and reproductive health & rights expert. RB is an associate professor at Radboud University Nijmegen Medical Center. EK, RB and JK designed the study. JK coordinated the data collection and performed the health economic analyses with RB. EK was responsible for the description and context of the six school-based sexuality education programs. JK, EK and RB wrote the manuscript. All authors read and approved the final manuscript.