Shekhar Saxena and colleagues summarize the recent WHO Mental Health Gap Action Programme (mhGAP) intervention guide that provides evidence-based management recommendations for mental, neurological, and substance use (MNS) disorders. The treatment gap for mental, neurological, and substance use (MNS) disorders is more than 75% in many low- and middle-income countries.In order to reduce the gap, the World Health Organization (WHO) has developed a model intervention guide within its Mental Health Gap Action Programme (mhGAP).The model intervention guide provides evidence-based recommendations developed with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology.This article presents the management recommendations for MNS disorders, with a link to the World Health Organization website where all the background material may be accessed.To our knowledge, this is a first exercise involving such an extensive and systematic evaluation of evidence in this area.Mental, neurological, and substance use (MNS) disorders are highly prevalent and are responsible for 14% of the global burden of disease expressed in disability-adjusted life years (DALYs) In order to reduce the gap, the World Health Organization (WHO) launched the Mental Health Gap Action Programme (mhGAP) to scale up services for people with MNS disorders, especially in LAMIC Evidence suggests that explicit, evidence-based, cost-effective packages of interventions can improve the processes and outcomes of health care when appropriately implemented. A package approach is being considered for a wide variety of health challenges such as the packages of interventions for family planning, safe abortion care, and maternal, newborn, and child health This article presents the management recommendations for MNS conditions, with a specific link (below) to the WHO website where all the background material may be accessed, including the evidence profiles and GRADE tables. Disseminating the recommendations through the scientific literature is important to generate feedback from health care professionals, policy–makers, and scientists. This will be helpful for re-defining the challenges ; re-evaluating the synthesized evidence; re-directing the development of the intervention guide for use in LAMIC; and re-defining any implementation strategies in LAMIC . In thishttp://www.who.int/mental_health/mhgap/evidence/en/index.html).WHO recommendations aim to be based on systematic reviews of the best available evidence and consideration of values, preferences, and feasibility issues from an international perspective and as reflected in relevant WHO policy documents and technical tools. Evidence profiles were developed for each of the priority conditions as well as a category labeled “other significant emotional or medically unexplained complaints.” Developing recommendations is a complex process that involves systematic review and assessment of the quality of evidence and balance of benefits and harms. In addition, explicit consideration of other issues such as value judgments, resource use, and feasibility, which are major considerations, need to be incorporated. Developing mhGAP recommendations within this framework highlighted several challenging and critical issues, including difficulties in formulating questions and rating outcomes, potential reproducibility and consistency problems, problems in incorporating observational evidence when evidence in the form of randomized trials was not available, and difficulties in defining how values, preferences, and feasibility issues should be considered, as the methodology for these aspects is much less developed The mhGAP guidelines highlight issues where uncertainty or controversy exists or areas where a change in policy or clinical practice is needed. One of the most challenging aspects of the whole process was that while for some mental health areas the background evidence was quite robust, for others the conventional way of synthesizing and evaluating the evidence revealed no or very poor quality evidence that was insufficient to make any recommendation. For example, some psychosocial interventions for individuals with psychosis or bipolar disorder are backed by few studies that provide inconclusive results. In such instances we took advantage of the GRADE methodology, which clearly recognizes that in addition to the evidence base, other aspects that are expected to inform the recommendations include consideration of values such as protection of human rights, feasibility and resource use, and GDG professional experience and their tacit knowledge. As previously noted, the added value of GRADE in these circumstances is that it requires GDG to transparently report that some recommendations are based on strong values and weak evidence Another challenging aspect was that while the main focus of WHO's mental health work was to develop recommendations for health care providers in low- and middle-income settings, most of the evidence comes from specialist settings of high-income countries. Despite this, we developed and applied instructions to overcome this compelling issue http://www.who.int/mental_health/mhgap/evidence/en/index.html). The readers of this article can also provide their feedback by posting comments online at PLoS Medicine. WHO will follow a systematic procedure for updating the recommendations following the same strict and transparent methodological process that was employed for their initial development.Maintaining and updating evidence-based recommendations is similarly relevant, as new evidence is continuously generated and accumulated in the international literature. Updating of mhGAP recommendations will not only be based on this new evidence, but also on feedback from health care professionals, policy-makers, scientists, and public health experts involved in implementing the guidelines. We also expect that revisions might include review of new questions or areas currently not covered by mhGAP guidelines, which might be identified as relevant questions to be addressed. In order to facilitate and encourage feedback, an electronically accessible web repository of recommendations and background material, including all the evidence profiles that were used to generate the recommendations, has recently been developed and will be regularly maintained and updated  for Mental, Neurological and Substance Use Disorders in Non-specialized Health SettingsThe scarce evidence base on how guidelines should be implemented must not weaken international efforts to develop and evaluate implementation plans The mhGAP-IG provides an integrated package of interventions for the management of priority MNS disorders. The reasons for integrated packages are the following: one, packages are an effective and efficient way of improving health service delivery. Two, delivery of interventions as a package is relatively cost-effective in terms of training and supervision. Many interventions and conditions go naturally together because they can be delivered by the same person at different times and because a limited range of interventions are applicable to a variety of conditions. Three, packages help concentrate scarce resources on the services that provide the best “value for the money”. We believe that the implementation of mhGAP guidelines and scaling them up would lead to a decrease in the burden of MNS disorders. For example, 7 million DALYs (11% of burden due to depression) are avertable with 50% coverage with interventions for depression (anti-depressant drugs [generic TCAs or SSRIs] plus brief psychotherapy) Although the mhGAP-IG is the principal clinical tool being used as part of the scaling up strategy of the mhGAP program in countries, accompanying training materials are being developed for capacity building. Program implementation using the mhGAP-IG has already started in four countries: Ethiopia, Jordan, Nigeria, and Panama Although research in the field of mental, neurological, and substance use disorders has significantly advanced in recent years, most of these advancements have been driven by the needs of health systems in the richest countries. To appropriately translate research findings into clinical and public health practices, it is critical to accelerate implementation research to evaluate interventions beyond the controlled conditions of research settings, and in the type of populations that suffer the largest proportion of the global burden of morbidity and mortality.Research areas that will help in scaling up mhGAP coverage include:Strengthening human resources, including capacity development and task shifting issuesImproving health financing to avoid out-of-pocket and point-of-care payments for health careSimplifying interventions so that they can be delivered with fewer resourcesIdentifying intervention delivery channels for greater reach and equity, and lower costs, including engagement of the non-government sectorImproving programme management and synergies between mental health and other programmes such as maternal or child health to overcome barriers to effective programme implementationMobilizing community resources for improved intervention delivery and increased health care utilizationSome examples of research questions are:How can mental health care using mhGAP be “added upon” the existing maternal health programmes?What are the specific barriers that hinder access to effective mental health interventions in rural populations?How can we use electronic and mobile technology to improve follow up for people with mental, neurological, and substance use disorders?How can the coverage, quality, and impact of mhGAP interventions improved by involving community health workers?What is the cost to the health system of scaling up mhGAP interventions when delivered through routine primary health care?


How is the performance of these new guidelines superior to the performance of existing guidelines to warrant/justify their introduction into the medical literature?The publication of Australian fetal alcohol spectrum disorder (FASD) diagnostic guidelines marks an important step forward in Australia’s efforts to prevent FASD. But do we need yet another set of FASD guidelines? At the 5th International FASD Conference, the ever growing number of FASD diagnostic guidelines was identified as a core area of concern by leaders in FASD worldwide. All agreed we need to strive to adopt a single set of guidelines. It is essential that FASD diagnosis advance to incorporate new knowledge and technology. But to date, the field of FASD has seen multiple sets of guidelines published that do not address the important question-J Popul Ther Clin Pharmacol20(3):e416–e467, 2013); the Australian guidelines relax the criteria to include the Rank 2 FAS facial phenotype. This relaxation of the criteria renders the facial phenotype NOT specific to prenatal alcohol exposure as confirmed in published empirical studies. If the facial phenotype is not specific to (caused only by) prenatal alcohol exposure one can no longer validly call the outcome PFAS. When one makes a diagnosis of FAS , one is stating explicitly that the individual has a syndrome caused by prenatal alcohol exposure. One is also stating explicitly that the biological mother drank alcohol during pregnancy and, as a result, harmed her child. These are bold conclusions to draw and are not without medical, ethical, and even legal consequences. So the question remains-Why go against the published empirical evidence and relax the PFAS facial criteria into the normal range?The Australian guidelines include FAS, PFAS and Neurodevelopmental Disorder-Alcohol Exposed (ND-AE). This latter group includes individuals with severe CNS abnormalities without the physical features of FAS. This is the group the 4-Digit-Code calls Static-Encephalopathy-Alcohol-Exposed (SE-AE). The criteria for FAS, PFAS, and ND-AE  are identical between the Australian and 4-Digit-Code guidelines with the exception of one very small, but very consequential difference in facial criteria for PFAS. The 4-Digit-Code requires a Rank 3 FAS facial phenotype for PFAS ( The publication of Australian fetal alcohol spectrum disorder (FASD) diagnostic guidelines  marks anHow is the performance of these new guidelines superior to the performance of existing guidelines to warrant/justify their introduction into the medical literature[FASD diagnostic guidelines have evolved over time since the term FAS was first coined in 1973 . Early giterature? At the The most recent guidelines introduced into the literature are the Australian guidelines . More acThe Australian guidelines include FAS, PFAS and Neurodevelopmental Disorder-Alcohol Exposed (ND-AE). This latter group includes individuals with severe CNS abnormalities without the physical features of FAS. This is the group the 4-Digit-Code calls Static-Encephalopathy-Alcohol-Exposed (SE-AE). The Australian guidelines chose not to use the term Alcohol-Related-Neurodevelopmental-Disorder  this moderate end of the FASD spectrum in the future.The Australian guidelines report the UW 4-Digit-Code could also be derived if desired“. Unfortunately, the only issue preventing clinicians from deriving a 4-Digit-Code that would be in complete compliance with the Australian guidelines is the relaxation of the PFAS facial criteria from a Rank 3 to a Rank 2. If the Australian guidelines required a Rank 3 face for PFAS, it would not only gain the specificity required to validate the use of the term PFAS, but the 4-Digit-Codes would derive 4-Digit clinical categories  that match the Australian clinical categories . If the 4-Digit-Code said it was FAS, so would the Australian guidelines. If the 4-Digit-Code said it was PFAS, so would the Australian guidelines. And if the 4-Digit-Code said it was SE-AE, so would the Australian guidelines ). Individuals who present with moderate CNS dysfunction  would receive 4-Digit-Codes documenting their outcomes, but in accordance with the Australian guidelines would not receive a label. This would seem a reasonable interim solution as Australians further assess how to handle this moderate end of the FASD spectrum. The authors report “Rochelle E Watkins, Elizabeth J Elliott, Janet M Payne, Colleen M O’Leary, Jane Halliday, Jane Latimer, Amanda Wilkins, Raewyn C Mutch, James P Fitzpatrick, Heather M Jones, Lorian Hayes, Heather D’Antoine, Sue Miers, Elizabeth Russell, Lucinda Burns, Anne McKenzie, Maureen Carter, Carol Bower.We have recently published recommendations for the diagnosis of fetal alcohol spectrum disorders (FASD) in Australia. These recommendations seek to address the known issue of underdiagnosis through supporting improved awareness of FASD and increased national capacity to respond to, and ultimately prevent, these disorders. We used an adaption approach to guideline development in recognition of existing diagnostic guidelines. Recommended national standard diagnostic criteria for Australia are based on the University of Washington (UW) 4-Digit Diagnostic Code and Canadian guidelines for FASD diagnosis. Our conclusions emphasise the importance of evaluation to improve the evidence-base for policy and practice.In her commentary on our recommendations, Astley has raised important questions about the publication of this work, including whether we need ‘yet another guideline’, and expressed concern about the recommended criteria for the diagnosis of partial fetal alcohol syndrome (PFAS). Guideline development is a recognised and widely used mechanism to influence clinical practice. The lack of guidelines for diagnosis in Australia has been identified as one of the factors contributing to the underdiagnosis of FASD. We sought to establish the basis for a standard national approach to diagnosis in Australia through the adoption or adaption of existing guidelines.Existing diagnostic guidelines lack agreement on all aspects of diagnosis, and the recommended diagnostic criteria for PFAS remain subject to debate. Although our recommended diagnostic criteria for PFAS in Australia based on the presence of 2 characteristic facial features differ from the UW guidelines, they lie within the range of criteria recommended by other guidelines. We sought peer review and publication of our findings to provide transparency and promote awareness of our work nationally and internationally. We support international collaboration to develop a common approach to diagnosis, and believe that this process will strengthen the evidence base for action globally.North America leads the world in the production of guidelines for the diagnosis of fetal alcohol spectrum disorder (FASD), and these provide a rich resource for those seeking to address and prevent the harm caused by prenatal exposure in other contexts. We recently published Australian recommendations for FASD screening and diagnosis  with theThe development of clinical practice guidelines has been identified as a critical activity at the national level to facilitate the delivery of effective health services . The majGuidelines are developed with the aim to influence practice. They need to be locally appropriate and integrated with strategies to facilitate their implementation. High quality guidelines developed at the international level can facilitate the development of locally appropriate guidelines for specific practice settings and, increasingly, local health providers are developing methods to integrate evidence in their own settings . The neeThrough the development of national recommendations for FASD screening and diagnosis we aimed to improve awareness of FASD and strengthen national capacity to identify, address and prevent the harms associated with alcohol consumption in pregnancy. Four key factors framed our approach: i) that there are no international consensus criteria for diagnosis; ii) that there is variation in diagnostic practices internationally ; iii) thA range of existing guidelines have been used for diagnosis in Australia. The Institute of Medicine criteria  were useWe used an adaption approach to development  in recogThere are considerable challenges in producing evidence-based guidelines for FASD screening and diagnosis, and disagreement between guidelines can arise for various reasons, not all of which imply that different recommendations are invalid . We deveIn the specific case of differences between the UW and Australian recommendations with respect to the facial features required for a diagnosis of PFAS, the panel recognised that the criteria for the diagnosis of PFAS remain subject to debate ,40. DiagRecommended facial criteria for the diagnosis of PFAS in Australia are consistent with existing guidelines ,18, and We believe that our work provides a credible basis for advancing practice in Australia and, consistent with the purposes of the scientific literature and the need for transparency in recommendation development, we sought peer review and publication of a research paper which documents the methods used to develop these recommendations. If, through seeking peer review and publication to promote national and international awareness of our goals and progress, this process highlights broader issues that confront the development of FASD guidelines internationally, then we have achieved more than we had hoped.The author declares that she has no competing interests.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2431/14/85/prepub


Differential item functioning (DIF) occurs when items in a measure perform in ways that are different for members of a target group when the different performance is not related to the individual's overall ability to be assessed. DIF may arise for a number of reasons but is often evaluated in order to ensure that tests and measures are fair evaluations of a group's abilities. Based on observations when administering the test, we developed the hypothesis that some items on the reading comprehension subtest of the Test of Functional Health Literacy (TOFHLA) might be differentially more difficult for older adults and the elderly due to its use of the cloze response format, in which the participant is required to determine what word, when placed in a blank space in a sentence, will ensure that the sentence is intelligible. Others have suggested that the cloze response format may make demands on verbal fluency, an ability that is reduced with the increasing age. Our analyses show that age-related DIF may present in a nearly one-half of reading comprehension items of the TOFHLA. Results of this measure in older persons should be interpreted cautiously. Healecisions .” It hasecisions , 4. One especially important finding in health literacy research has been the fact that racial and ethnic minorities and the elderly perform at lower levels on several measures of health literacy compared to the general population , 6. One  Although studies have often treated health literacy as a unitary characteristic of the persons assessed, studies have used a number of different measures to assess it . It is nAnother widely used measure, the Rapid Estimate of Adult Literacy in Medicine, or REALM , assesse In a previous study, we used the TOFHLA with elderly patients who were being treated with medications for memory problems , 16. In differential item functioning (DIF) [ Other authors have suggested that the cloze format may be difficult for older adults due to its demands on cognitive abilities known to decline with increasing age, including verbal fluency working memory and psychomotor speed . Furtherng (DIF) . DIF is ng (DIF) , the res Data for this study were drawn from a study of cognition and medication adherence in persons treated for HIV . ParticiAs part of a battery of measures, the reading comprehension portion of the TOFHLA was administered. This measure comprises three health-related paragraphs of increasing reading difficulty, beginning with instructions on how to prepare for a radiographic study and concluding with an informed consent for a surgical procedure. Words are removed from sentences with a blank substituted, and possible correct options are listed below each blank. The total number of responses for the all paragraphs is 50. Participants were tested according to the standard directions for the measure  and werehttp://www.psych.mcgill.ca/misc/fda/downloads/testgraf/), a package that is freely available for download [beta, for each item. Based on extensive simulation modeling, Zumbo and Witarsa [http://www.itemanalysis.com/), to estimate item difficulties, standard deviations, and discriminations .  Sample sizes required for stable estimates via parametric item response theory (IRT) are large. Most experts suggest that sample size should be in the range of 1,000 . Becausedownload . In addi Witarsa  have proWe divided our sample into two groups, those with ages less than and those with ages equal to or greater than 45 years. This cut point was chosen as it provided reasonably similar sample sizes for each group and lies in the age range related to both evidence of cognitive aging  and lowe Statistics providing a characterization of the sample are presented in P < 0.01 cut point provided by Zumbo and Witarsa are italicized [ Results of item analyses are presented in alicized . It can  Results of these analyses suggest the existence of substantial age-related DIF in the reading comprehension subtest of the TOFHLA. To the best of our knowledge, this is the first study to evaluate at an item level the influence of age on TOFHLA scores. Based on our observations of participants in an earlier study, we investigated the possibility that the response format of the TOHFLA might have an influence on older adults' performance independent of their actual levels of health literacy. Our results suggest that this may be the case. The implication of this finding is that at least a portion of the difference in health literacy associated with age on the TOFHLA may be the result of DIF rather than actual differences in health literacy. It should be noted that some studies have not found age-related differences in health literacy when using a measure that does not use the cloze response format . One st Limitations of this study should be acknowledged. Our sample included only persons treated for HIV infection, potentially limiting the extent to which these findings can be generalized to other populations of older adults. Although our participants may have had HIV-related cognitive deficits that could have affected their performance on the S-TOFHLA, it is also likely that they would have had age-related changes in cognitive function. The dual effects of HIV infection and aging on cognitive function (presumably the basis for finding DIF on the S-TOFHLA) are difficult to distinguish; studies of the issue have suggested that both aging and HIV have an impact on cognition  while atWhile it thus might appear that age-related deficits in health literacy may be related to the response format of the measure used to assess it, it must be noted that other measures have found age-related differences in health literacy. Although  it is difficult to know the precise format of responses on the measure, the National Assessment of Adult Literacy found significant deficits in health literacy among older adults. Haun et al. found age-related differences in performance on the S-TOFHLA and a self-report measure of health literacy, the BRIEF , but not


Guidelines are important tools that inform healthcare delivery based on best available research evidence. Guideline use is in part based on quality of the guidelines, which includes advice for implementation and has been shown to vary. Others hypothesized this is due to limited instructions in guideline development manuals. The purpose of this study was to examine manual instructions for implementation advice.We used a directed and summative content analysis approach based on an established framework of guideline implementability. Six manuals identified by another research group were examined to enumerate implementability domains and elements.Manuals were similar in content but lacked sufficient detail in particular domains. Most frequently this was Accomodation, which includes information that would help guideline users anticipate and/or overcome organizational and system level barriers. In more than one manual, information was also lacking for Communicability, information that would educate patients or facilitate their involvement in shared decision making, and Applicability, or clinical parameters to help clinicians tailor recommendations for individual patients.Most manuals that direct guideline development lack complete information about incorporating implementation advice. These findings can be used by those who developed the manuals to consider expanding their content in these domains. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated. New approaches for guideline development and implementation may need to be developed. Use of guidelines might be improved if they included implementation advice, but this must be evaluated through ongoing research. Research, practice, and policy in the healthcare sector focus on improving the organization, delivery, and outcomes of care. Critical to achieving these objectives is the need for guidance based on currently available knowledge generated through research. This has led to an emphasis on evidence syntheses such as guidelines that are defined as ‘systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances’ ,2. WhileNumerous factors influence whether and how guidelines are used. Some of these are extrinsic factors, such as: the nature of the newly recommended practice or technology itself; characteristics of healthcare providers; organizational capacity to collect, adapt, share, and apply evidence; and system-level environmental factors -10. Becaet al. evaluated 279 guidelines published from 1985 to June 1997 by 69 different developers, and found that mean overall adherence to standards for guideline format, methods, identification and summary of evidence, and formulation of recommendations was 43% [To promote the development and use of high quality guidelines, the Appraisal of Guidelines Research and Evaluation (AGREE) instrument was developed by an international collaboration of guideline development experts . A revis was 43% . When thet al. proposed that variability in manuals describing guideline development methods may be causing the mismatch between standards for guideline quality and the actual quality of guidelines [Turner idelines . They idet al. found that four of six manuals described instructions for developing implementation advice that could be included in guidelines. However, the framework they used to examine manuals was based largely on the AGREE instrument, supplemented by a literature search for other key methods related to the development of guidelines. The framework consisted of 14 elements organized in four domains: preparing for guideline development, systematically reviewing the evidence, drafting the guideline, and reviewing the guideline. Only one element pertained to implementation, and they only looked for presence or absence of any implementation information, and not the details of instructions for developing implementation advice.It is now well recognized that simply producing guidelines does not itself lead to use, and that active implementation is needed to encourage their uptake. The Knowledge to Action Cycle offers a framework for promoting the use of guidelines . It recoet al., guidelines may offer little support to users for implementing the recommendations because guideline development manuals lack instructions for developing such implementation advice. To confirm this and more thoroughly describe the nature of instructions for developing implementation advice in guideline development manuals, the purpose of this research was to re-examine the manuals sampled by Turner et al. according to our framework of implementability, which reflects guideline components shown by research to be associated with intent to, and actual use of guidelines. Use of a more detailed framework serves to pinpoint specific implementation information that may be lacking, thereby informing future updating of such manuals, or development of adjunct products.We investigated how to make guidelines more implementable by modifying their content and format . We firsContent analysis describes ideas in written, verbal, or visual communication to generate, extend, or validate a framework or model . It invoet al. because they used a comprehensive strategy to search for and identify the manuals, including the MEDLINE database, various guideline web sites, web sites of known guideline developers, a general search of the Internet using Google, and consultation with members of an evidence-based healthcare email list. They included manuals published in English language, produced by international organizations responsible for guideline development, and supporting the development of evidence-based guidelines, and excluded manuals issued by specialty societies for specific conditions.The unit of analysis is the guideline development manual. These were sampled by convenience. We examined the same manuals identified by Turner Full-text manuals were retrieved from organization web sites, which were checked once again at the time of writing this manuscript to ensure the most recent versions were analyzed. A data extraction form reflecting the implementability framework domains and elements was developed. It was used to extract information about the presence or absence of each element anywhere within the guideline development manuals, plus producer, date of publication, number of pages, stated objective of the manual, implementation context (the stated purpose or setting of implementation), and how the implementation content was organized within the manual. Two individuals independently extracted data, and a third resolved differences and tabulated the data.Tabulated findings were examined to describe general features of each manual, and enumerate the presence of implementation instructions reflecting elements from the implementability framework.Manual features are summarized in Table Implementability domains and elements addressed in manuals are enumerated in Table et al. [et al. that the manuals were similar but lacked sufficient detail. We did not identify trends in manual purpose, implementation context, or content by type of guideline developer. Use of a more comprehensive framework reflecting the multiple steps of implementation revealed specific topics not addressed. Most frequently this was Accomodation, or information that would help guideline users anticipate and/or overcome organizational- and system-level barriers. This may include: information about equipment or technology needed; industrial standards; policies governing their use; type and number of health professionals needed to deliver services; education, training, or competencies needed by staff to deliver services; anticipated changes in workflow or processes during or after adoption of guideline recommendations. In more than one manual, Communicability information was lacking, or resources that physicians and/or patients could use for education or to facilitate shared decision making, as was Applicability information, or clinical parameters to help physicians tailor guideline recommendations for individual patients.We examined the content of the same guideline development manuals reviewed by Turner et al.  using a While directions for creating Applicability information were absent in guideline development manuals, our previous work found that 90% of guidelines we examined did include clinical considerations by which to individualize recommendations . This isThe guideline enterprise is challenged to keep up with these changes and modify the traditional guideline development and implementation strategy. This may require consideration of different knowledge sources and evidence, involvement of different types of experts in the guideline development and implementation process either in staged fashion or as multidisciplinary or interprofessional teams, and partnerships between those who synthesize and interpret clinical effectiveness data and those who plan and/or undertake implementation. We know that guideline implementation is complex, and many developers lack the mandate and/or resources to implement the guidelines they produce ,33, so aInterpretation of these findings may be confounded by several issues. Not all existing guideline manuals were examined. We instead relied on the comprehensive search strategy applied by another research team to identify our sample. They chose to examine six manuals that were produced by major international guideline development organizations, and of a general nature and therefore of broad relevance. We did search for and examine newer versions of those manuals if they were available. We also ran an updated search for other general guideline development manuals, and found only one published by the Canadian Medical Association . For conOur implementability framework, while formulated based on expressed needs among various types of guideline users and studies positively associating these features with actual guideline use, has not been thoroughly validated. We have yet to conduct experimental studies that would test whether inclusion of information or tools reflecting these implementability domains leads to guideline implementation, use, and beneficial healthcare outcomes. We also need to assess how different types of users  would interpret and use implementability tools. First, however, we need to develop implementability tools because our most recent analysis of guidelines (not yet published) found that few contained such tools. In advance of such development, we are further validating the framework with input from international guideline developers, implementers, and researchers. Despite the need for further validation, our framework offers a more comprehensive way to evaluate implementation instructions in guideline development manuals that reflects the emerging, more complex multi-step paradigm of guideline implementation. Therefore, this study provides a more detailed evaluation than previously published on whether differences in intrinsic qualities of guidelines may be due to differences in the content of development manuals, and more precisely how manuals could be refined to facilitate the development of guidelines that provide users with implementation advice. These findings can be used by those who developed the manuals to consider expanding their content. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated.Another limitation—not necessarily of our study, but in the evidence on guideline development and implementation—is the lack of criteria by which to assess the quality or actionability of instructions for developing guidelines or guideline implementation advice . By usinIn conclusion, most manuals that direct guideline development lack complete information about incorporating implementation advice for elements considered important by health professionals and associated with guideline use. These findings can be used by those who developed the manuals to consider expanding their content in these domains. It can also be used by guideline developers as they plan the content and implementation of their guidelines so that the two are integrated. However, to embrace the emerging expanded paradigm of guideline development and implementation, new approaches may need to be considered, including use of different knowledge sources and evidence, involvement of different types of experts, and partnerships between those who synthesize and interpret clinical effectiveness data and those who plan and/or undertake implementation. Who bears responsibility for implementing guidelines, what resources are required to support implementation, and who will provide those resources must also be considered. Use of guidelines might be improved if they included implementation advice, but this must be evaluated through ongoing research.The authors declare that they have no competing interests.ARG conceived the study and its design, acquired funding, and coordinated its conduct. MCB contributed to study design, and participated in data interpretation. All authors read and approved the final manuscript.


In both developing countries and in relation to cultural minorities there have been calls to scale up mental health services and for evidence-informed policy and practice.The evidence based medicine movement has had a major influence in improving practice. However, implementation of this approach has some major difficulties. One that has been neglected is the situation where there is no relevant evidence. This situation is more likely to occur for healthcare decisions in developing countries or for cultural minorities within developed countries, because resources do not exist for expensive research studies.Consensus methods, such as the Delphi process, can be useful in providing an evidence base in situations where there is insufficient evidence. They provide a way of systematically tapping the expertise of people working in the area and give evidence that is readily applicable for a particular country and culture. Although consensus methods are often thought of as low in the hierarchy of evidence, consensus is central to the scientific process. We present four examples where the Delphi method was used to assess expert consensus in situations where no other evidence existed: estimating the prevalence of dementia in developing countries, developing mental health first aid guidelines in Asian countries, mental health first aid guidelines for Australian Aboriginal people, and modification of the concept of 'recovery' for Australian immigrant communities.Consensus methods can provide a basis for decision-making and considered action when there is no evidence or when there are doubts about the applicability of evidence that has been generated from other populations or health system settings. Mental disorders are common in all countries, with considerable variation in reported prevalence across countries, even when the same methods for estimation of prevalence are used . The treOver the past 30 years, the evidence based medicine movement has had a major influence in improving practice. Evidence based medicine involves "integrating clinical expertise with the best available clinical evidence derived from systematic research" . This moDespite its many successes, the evidence based medicine movement has encountered some significant barriers in changing clinical practice, including the difficulty that practitioners have in finding and interpreting the evidence, and in applying it in practice, due to organisational barriers, patient adherence and lack of quality practitioner education . There hHere we discuss an obvious, but neglected, barrier to evidence based practice--situations where there is no evidence . This baDeveloping countries have limited resources to support research . TherefoThe development of a locally relevant evidence base using expert consensus is a valuable approach where other evidence is unavailable. While developing countries may not have the resources to carry out randomised controlled trials, population surveys, cohort studies or health service evaluations , they doAs well as being feasible with limited resources, consensus methods overcome some of the limitations that have been identified with evidence based medicine. They embody the principle of practice-based evidence, provide results that are relevant to the local population and culture, and are readily implementable within the healthcare system. This approach fits with the call to improve practice by capitalising on accumulated practical experience and using this to develop better interventions.The Delphi consensus method has been widely used in disparate fields to inform: policy-making ,33; desiThere are several possible objections to the adequacy of expert consensus as a source of evidence. The evidence based medicine movement has been associated with a hierarchy of levels of evidence. Typically, the strongest form of evidence is held to be a meta-analysis of randomised controlled trials, with expert consensus ranked at the bottom of the hierarchy. However, while systematic reviews and meta-analyses are held to be the highest form of evidence, there is little agreement on the reporting of a critically important component of the systematic review process - the search methods used in carrying out systematic reviews . Even whIf expert consensus is so weak, should we be proposing it as a strategy for contributing to informed decision-making? An important counter-question to ask is: how were some types of evidence assigned to be higher in the hierarchy than others? The answer is "expert consensus". Expert consensus is seen as adequate to validate hierarchies of evidence or indeed other components of the evidence based medicine enterprise, such as the CONSORT Statement for reporting randomised controlled trials  and prinMore generally, consensus has an important role in the scientific process. New theories gain ground as more members of the scientific community see a new theory as giving a better account of the evidence than older ones . The rolA paradox in recommending consensus in order to make progress is that consensus may hinder progress and promote no longer useful or even harmful practices. Consensus is an expression of the prevailing political, social, cultural, scientific world view of the time. There may, for example, be a broad consensus within society with which scientists do not agree, or there may be a consensus among experts that produces undesirable outcomes . The valper se is weak or strong, because it is integral to how science works, but rather what foundation of evidence the consensus rests on. A consensus based on a set of randomised controlled trials [The issue is not whether consensus d trials  may welld trials . Expert To illustrate the use of expert consensus methods in situations where there is no evidence, we describe four projects that have used the Delphi process to fill evidence gaps in developing countries and with cultural minorities.Studies of the prevalence of dementia are major research undertakings. Estimates of prevalence rates are available from well-conducted studies in most developed countries, but for the rest of the world there is limited or no evidence. To overcome this lack, Alzheimer's Disease International carried out a Delphi study with a panel of 12 international experts . The panMental health first aid is the help provided to a person who is developing a mental health problem or is in a mental health crisis. A Mental Health First Aid training course has been developed in Australia and has spread to many other countries . Mental A related example concerns Aboriginal Australians, who comprise less than 3% of the Australian population, but are a priority for action because they are disadvantaged in both physical and mental health. A Mental Health First Aid training program has been developed for Aboriginal and Torres Strait Islander peoples and found to be culturally acceptable . HoweverMental health policy, particularly as it relates to the mental health NGO sector that provides rehabilitation, social support, housing, employment and related services, increasingly requires that recovery principles should underpin service design and delivery. However, the origins of the recovery movement are in western Anglophone countries, and recovery principles reflect the cultural commitment of these countries to values such as privacy and confidentiality, individual autonomy, and self-determination. The relevance of these underlying values to many immigrant communities in Australia, which tend to place greater value on interdependence, collective decision-making, and authority of the senior member of the family, has been called into question, therefore challenging whether the current recovery construct can be applied in a multicultural society. This study  used theEvidence is most important as a basis for decision-making and action, such as medical treatment of an individual, decisions about health policy, or actions such as investment in one rather than another form of service delivery. It is clear that such decisions will be made and actions embarked upon whether or not there is evidence or whether the decision-maker or practitioner is familiar with such evidence as does exist. Consensus methods can provide a basis for decision-making and considered action when there is no evidence or when there are doubts about the applicability in a particular setting of evidence that has been generated from other populations or health systems. The issue is not whether consensus methods provide evidence that is as good as other ways of generating evidence, but whether the evidence generated by using such methods is better than no evidence or inapplicable evidence.Beyond questions of the availability or quality of evidence, expert consensus methods have a particular advantage when applied in fields that are neglected and that require urgent action, such as mental health system development in low-income countries and for immigrant and refugee communities. The process of identifying consensus shifts the focus away from areas of disagreement, which frequently are many, to areas of agreement. If key stakeholders, such as government decision makers, service users, as well as clinicians and researchers, are involved in the expert panel considering a particular issue, this can facilitate informed decision-making and action.The authors declare that they have no competing interests.HM and AJF contributed equally to the conception and writing of this paper. Both authors read and approved the final manuscript.


AVPR1A) and social cognition in chimpanzees. Studies have shown that chimpanzees are polymorphic for a deletion in a sequence in the 5′ flanking region of the AVPR1A, DupB, which contains the variable RS3 repetitive element, which has been associated with variation in social behavior in humans. Results revealed that performance on the social cognition task was significantly heritable. Furthermore, males with one DupB+ allele performed significantly better and were more responsive to socio-communicative cues than males homozygous for the DupB- deletion. Performance on a non-social cognition task was not associated with the AVPR1A genotype. The collective findings show that AVPR1A polymorphisms are associated with individual differences in performance on a receptive joint attention task in chimpanzees.Despite their genetic similarity to humans, our understanding of the role of genes on cognitive traits in chimpanzees remains virtually unexplored. Here, we examined the relationship between genetic variation in the arginine vasopressin V1a receptor gene ( At approximately 6 months of age, typically developing human infants begin to respond to socio-communicative cues of conspecifics such as gaze and following pointing cues. Some have suggested that the development of responding to joint attention cues is important for the emergence of language345891112Not only do children respond to gaze and manual pointing, but studies in nonhuman primates, including monkeys but particularly great apes, have shown some similar findings. A number of nonhuman primate species will follow the gaze of human experimenters or conspecifics15161819In studies of both human and nonhuman primates, there are considerable individual differences in receptive joint attention (RJA) performance; however, the potential biological factors that underlie this variation are poorly understood10192224Specifically, we assessed RJA performance in a sample of captive chimpanzees for which there is a well-documented pedigree. In the initial analysis, we capitalized on the existence of the pedigrees to perform a quantitative genetic analysis in order to estimate heritability in RJA performance of the chimpanzees. Similar methods have recently been employed in several nonhuman primate species to determine potential genetic contributions to individual differences in behavioral traits such as a novelty-seeking27AVPR1A) on RJA performance in the chimpanzees. The AVPR1A gene was selected as a candidate gene for several reasons. Notably, vasopressin is a neuropeptide with multiple physiological functions that has been strongly implicated in the development and evolution of complex social relations and cognition in mammals30Avpr1a). There are both individual and species differences in Avpr1a microsatellite structure, which has been associated with variation in AVPR1A expression in the brain, pair bonding and other dimensions of social behavior in voles, particularly males32333435In addition to the quantitative genetic analysis, we also assessed the potential influence of a vasopressin receptor gene 4-TT-(CT)8-(GT)24 repeat that is 3625 bp upstream of the transcription start site. Homozygosity in allele 334 of RS3 in men (but not women) is associated with variation in pair bonding behavior, as measured by traits such as relationship quality, perceived marital problems, marital status, and spousal perception of marital qualityAVPR1A mRNA in postmortem hippocampal tissue compared to shorter allelesAVPR1A expression in specific brain regions, as previously demonstrated in rodentsThese studies parallel findings in humans that describe an association between microsatellite variation in the AVPR1A gene because there is a common indel (insertion/deletion) resulting in a complete deletion of the RS3 sequence in roughly 65% of chimpanzees4220–26 microsatellite, known as STR1. The second block, DupB, spans −3382 to −3729 bp and contains the complex microsatellite, RS3 ((CT)6–14(GT)8–24). Chimpanzees are polymorphic for the presence of the RS3-containing DupB region (DupB+), leading to a 357 bp difference between the DupB+ and DupB− alleles~65% of chimpanzees makes this species ideal for assessing the potential role of the AVPR1A gene, and more specifically RS3, on social behavior and cognition. Finally, it has been previously reported that DupB+/− male chimpanzees are rated as more dominant than DupB−/− males+/− males might be more dominant because they are more sensitive to subtle, non-verbal social cues produced by conspecifics that would presumably result in them responding more appropriately than those who are less sensitive (DupB−/−). This, in turn, might facilitate the formation of reciprocal relationships among group members, thereby increasing the social support needed to raise social status within the group as well as potentially attract more mates44Additionally, chimpanzees are an excellent model to explore the functional role of the RS3 polymorphism in the AVPR1A DupB region containing RS3 and also tested them on a RJA task, originally developed for use with typically developing human children and those at risk for developmental disorders, such as autismAVPR1A expression contributes to individual differences in RJA performance, then heterozygous DupB+/− chimpanzees would perform better than homozygous DupB−/− genotype individuals.To this end, we genotyped a sample of captive chimpanzees for the z = 12.22, N = 232, p < .001). Eighty-two percent of the chimpanzees (N = 190) performed better on the experimental compared to baseline test whereas the remaining18% of the chimpanzees (N = 42) responded equally or did worse on the experimental compared to the baseline tests. The distribution of performance scores for males and females is presented in In the initial analysis, we examined whether the chimpanzees were more likely to respond in the experimental compared to baseline conditions as a means of assessing the effectiveness of this test to elicit RJA responses and to rule out the possibility that the apes were randomly responding. For this analysis, we performed a Wilcoxon signed rank test based on the summed scores for the two baseline and two test trials and a significant difference was found between conditions . We next ran the quantitative genetic analysis again but added age, colony , sex , AVPR1A genotype  and the sex X AVPR1A interaction terms as covariates in the model. With these variables in the model, heritability in RJA performance was reduced and only borderline significant . The covariates sex, AVPR1A and the interaction between sex and AVPR1A genotype were all significant and accounted for 4.9% of the variance. Because the higher order sex by AVPR1A interaction was significant, we focused the post-hoc interpretation of this finding. As can be seen in +/− males needed significantly fewer social cues than DupB−/− males to elicit an orienting response. No significant differences were found between DupB+/− and DupB−/− females.We next performed the initial estimation of hAVPR1A genotype  were fixed factors. This analysis revealed a significant two-way interaction between sex and AVPR1a genotype F = 6.217, p < .02. Post-hoc analysis indicated that DupB+/− males needed significantly fewer social cues than DupB−/− males to elicit an orienting response. No significant differences were found between DupB+/− and DupB−/− females. No other significant main effects or interactions were found.As a secondary analysis, we performed a more traditional analysis of variance with the RJA score serving as the dependent measure while colony  sex  and AVPR1A effects on RJA were specific to that task or were also evident for other non-social tasks. Thus, we examined the effects of AVPR1A genotype and sex on SM and RJA performance using MANOVA. RJA and SM performance were the dependent measures while sex  and AVPR1A  were the independent variables. The MANOVA revealed a significant two-way interaction between sex and AVPR1AF = 2.87, p < .05. Subsequent univariate F-test indicated that the significant interaction between sex and AVPR1A was specific to the RJA F = 4.40, p < .04 and not the SM task F = .530, p = .37. The mean RJA and SM performance in DupB−/− and DupB−/+ male and female chimpanzees are shown in Recall that this task was administered only to the 71 YNPRC chimpanzees and was used to evaluate whether the observed AVPR1A gene and this was particularly the case for male but not female chimpanzees. Our analysis suggests that variation in AVPR1A RS3 explains approximately 5% of the variance in RJA performance, which is a fairly robust effect size for a single gene's impact on behavior. While our data do not suggest that variation in AVPR1A contributes to a large proportion of individual variation in RJA, the data do suggest that AVPR1A plays a role in the socio-communicative skills of chimpanzees. Indeed, these data are the first evidence of a genetic basis for explaining individual differences in RJA performance in any species, including humans .Three findings emerged from this study. First, consistent with previous studies, chimpanzees orient to human communicative cues including gaze and pointing and do not in their absence. Second, individual differences in RJA performance were significantly heritable, suggesting that genetic variation explains 25% of the variation in RJA performance. Third, after controlling for genetic relatedness, RJA performance was linked to variation in the s but see. It shous but see. FurtherAVPR1A gene influences the sensitivity in perception of socio-communicative cues by male but not female chimpanzees. As RS3 has been linked to AVPR1A mRNA expression in human post-mortem tissue, and promoter activity in transcription assaysAVPR1A gene on social cognition has not been previously reported in human or nonhuman primates but is consistent with studies demonstrating that vasopressin has selective effects on male social behavior in voles, including pair bondingThe overall results suggest that the AVPR1A has been linked to variation in relationship quality in men but not women+ allele with RS3, which is similar to the human AVPR1A gene structure, are more sensitive, and respond better to, socio-communicative cues than those individuals homozygous for the more common DupB− allele, which is missing the RS3 microsatellite.In humans, the RS3 in the AVPR1A on RJA performance in the chimpanzees. Though we compared both RJA and SM performance within the YNPRC cohort, data on additional non-social cognitive measures in all the apes would be more desirable. Notwithstanding, we do believe that the effect of the AVPR1a polymorphism reported here likely does reflect its role in subject abilities to attend to socio-communicative cues rather than more general intellectual skills for two reasons. First, as noted above, there is a fairly large body of research implicating the AVPR1A gene in a variety of social behaviors in mammals and the results reported are consistent with these observations. Further, some preliminary analyses examining the effect of the AVPR1A gene on a different non-social cognitive task in a subset of chimpanzees in this study revealed little evidence that it influences performance. Specifically, within the UTMDACC and YNPRC cohort, we have previously assessed cognition for a variety of different problem solving and motivational tasks. For instance, we provided 145 chimpanzees that were genotyped for the AVPR1A with coconuts and measured whether they were able to successfully learn to open them within three separate test sessions. Performance on the coconut task was found to be heritable  but AVPR1A genotype was not significantly associated with performance with 67% (43/59) and 66% (46/76) percent of the DupB−/+ and DupB−/− chimpanzees able to solve the problem. Separate analyses of the association between AVPR1A genotype and success in opening a coconut for males and females showed no significant effects for either sex. Among the males, 74% (32/43) and 69% (11/16) percent of the DupB−/− and DupB −/+ individuals could solve the task. Likewise, in the females, 59% (31/53) and 65% (15/23) of the DupB −/− and DupB −/+ subjects successfully opened the coconut. Thus, like the SM task, these data do not support the notion that the AVPR1A is generally involved in problem solving abilities.There are at least three limitations to this study. First, we examined the heritability and role of the +/+ genotype for inclusion in the genotype analysis. As noted in the methods section, we identified only 9 chimpanzees with the DupB+/+ genotype including 8 females and 1 male. The limited number of DupB+/+ chimpanzees is consistent with previous findings by Donaldsen et al.−/− and DupB−/+ individuals but how the DupB+/+ individual would perform on the RJA task in comparison is unknown and will require additional data collection from another sample of chimpanzees.A second, limitation was the inability to obtain a reasonable sample of chimpanzees with the DupB AVPR1A is expressed in the chimpanzee brain and how this might interact with neurobiological measures to explain individual differences in RJA performance. For instance, Hopkins and Taglialatela51AVPR1A may be expressed in the anterior cingulate cortex and superior temporal gyrus (among other regions), as has been reported in rhesus macaquesin vitro assays in relation to microsatellite variability in cell culture for different cell types would be potentially fruitful. These techniques have been used in humans and rodents3541Third, this study showed that receptive joint attention was associated with the absence or presence of the DupB+ in our chimpanzee sample. As such, this should be considered a first step in a larger attempt to understand the role of the AVPR1A on socio-communicative and cognitive process in chimpanzees; however, we recognize that the variation in DupB+ may be in disequilibrium with another functional elements and we cannot rule this out based on the findings of this study. We also know of no data on where AVPR1A in relation to finite set of tasks, we have attempted to limit the potential for Type I error and thereby minimize reporting a potential false positive. Furthermore, as noted above, because we found no significant main effects or interactions for the colony variable in relation to sex or AVPR1A genotype, we believe this represents an independent replication of the results between two chimpanzee samples. Notwithstanding, additional studies are needed to determine the consistency of the results reported here in other chimpanzee populations.Lastly, as is the case in any genotype-phenotype study, our findings should be interpreted within the context of potential false positive results. By limiting our analyses to a single dichotomous polymorphism in the +/− males were ranked as more dominant than DupB −/− males and they hypothesized that these individuals, in turn, may be dominant because of their sensitivity in perceiving and responding to different social cues. Based on these previous findings, combined with those reported here, it leads to the hypothesis that significant associations would be found between social rank and RJA performance in chimpanzees. It further suggests that the association between social rank or dominance and RJA performance would be mediated by variation in the DupB region in chimpanzees.In a larger evolutionary context, the findings reported here are consistent with the hypothesis that higher ranking chimpanzees may be more sensitive to interpreting and responding to social cues than lower ranking individuals. Recall that Hopkins et al.AVPR1A gene may be a risk factor for the development of autism-spectrum disorder (ASD)5513AVPR1A gene should be further investigated as a potential genetic marker for explaining individual differences in RJA performance and potentially socio-cognitive endophenotypes of ASD subjects, particularly males.Finally, it has been hypothesized that the Pan troglodytes verusThe quantitative genetic analysis for the RJA task was based on a sample of 232 adult or sub-adult chimpanzees and all individuals were residing at either the Yerkes National Primate Research Center  or The University of Texas M. D. Anderson Cancer Center . Based on previous analyses of the composition of subspecies of chimpanzees in captive United States populations, we assumed that at least 95% of our sample was represented by gazed for 5 sec toward a spot over and behind the subject's head. E then returned to a neutral position for 10 seconds. If the subject overtly oriented and/or looked behind them and the directed line of sight of E within the 15 second test time-frame, they receive a score of 3 for the trial and the session was ended. If the subject did not orient within the 15 second test time-frame, the subject was tested on Step 2. In Step 2, E looked at the same spot as in Step 1 but this time both gazed and pointed to the spot behind the subject for 5 seconds  or minimum score of 0 . Similarly, for the baseline trials, the subjects were classified as passing or failing each trial resulting in a range of scores from 0 (never looked with no cue given) to 2 (looked both times when no cue was given). For the second performance characterization measure, we considered the raw scores on the two test trials, which could range from 6  to 0 . Scores between 1 and 6 indicated some variation in the number of social cues needed to elicit an orienting response with higher scores indicating the need for fewer socio-communicative cues. For the SM task, the percentage of correct trials (out of 6) was the dependent measure.2) is the proportion of total phenotypic variance that is attributable to all genetic sources. Total phenotypic variance is constrained to a value of 1; therefore, all non-genetic contributions to the phenotype are equal to 1 - h2. Many of the chimpanzees in each colony are related and this allowed for an analysis of heritability using quantitative genetics based on the entire pedigree. To estimate heritability in RJA performance, we used the software package SOLAR6162AVPR1A polymorphism on RJA performance when controlling for relatedness among the individuals. Similar methods have been used to examine, for example, whether polymorphisms in the 5-HIAA serotonin transporter gene influence temperament in genetically related rhesus monkeys2) in RJA performance. When then subsequently examined changes in h2 when adding colony, sex, AVPR1A genotype and the interaction between these two variables as covariates in the analysis. No change or reductions in h2 after inclusion of the covariates indicates that the specific genes have a significant impact on the phenotype independent of the relatedness among the individualsTotal additive genetic variance  as described by Donaldson et alAVPR1A DupA/B region using the primers and conditions reported in previous studies with slight modifications~900 bp, while the DupB minus allele was ~570 bp long, and genotypes were visually assigned−/− and 66 DupB−/+ and 9 DupB+/+ chimpanzees. We would note that we identified 9 DupB+/+ animals, however the sex ratio was severely skewed for unknown reasons . We would further add that the distribution of DupB−/−, DupB −/+ and DupB +/+ from a sample of 47 wild caught chimpanzees genotyped by Donaldson et al.2 = 0.51, p = .75. Thus, our genotype distribution does not deviate from Hardy-Weinberg equilibrium. Because we had so few and such a skewed sex distribution of DupB+/+ apes, they were omitted from the analysis presented here. However, it should be noted that when we combined DupB+/+ animals with DupB−/+, the statistical findings were not changed.Each individual was genotyped for the W.D.H. designed, analyzed and wrote the manuscript. A.K. and L.Y. assisted in data collection, data analysis and writing of the manuscript. L.R. and J.S. assisted in data collection and S.S. assisted in writing. All authors have read the manuscript and have agreed to the content.


Low health literacy is generally associated with poor health outcomes; however, health literacy has received little attention in multiple sclerosis (MS).The aim of this study was to investigate the health literacy of persons with MSusing the North American Research Committee on Multiple Sclerosis (NARCOMS) Registry.In 2012, we conducted a cross-sectional study of health literacy among NARCOMS participants. Respondents completed the Medical Term Recognition Test (METER) which assesses the ability to distinguish medical and nonmedical words, and the Newest Vital Sign (NVS) instrument which evaluates reading, interpretation, and numeracy skills. Respondents reported their sociodemographic characteristics, health behaviors, comorbidities, visits to the emergency room (ER), and hospitalizations in the last 6 months. We used logistic regression to evaluate the characteristics associated with functional literacy, and the association between functional literacy and health care utilization.P<.001). After adjustment, low literacy on the METER was associated with ER visits  and hospitalizations . Findings were similar for the NVS.Of 13,020 eligible participants, 8934 (68.6%) completed the questionnaire and were US residents. Most of them performed well on the instruments with 81.04% (7066/8719) having functional literacy on the METER and 74.62% (6666/8933) having adequate literacy on the NVS. Low literacy on the METER or the NVS was associated with smoking, being overweight or obese (all In the NARCOMS cohort, functional health literacy is high. However, lower levels of health literacy are associated with adverse health behaviors and greater health care utilization. The elements of general literacy include the knowledge and skills to comprehend and use written information, to locate and use information captured in documents such as maps, and numeracy. Health literacy builds on these concepts , and refA growing literature suggests that lower health literacy is associated with higher rates of health care utilization and mortality, lower rates of health promoting activities, lower adherence to therapy and less successful disease control ,3-7. DesWe aimed to investigate the health literacy of persons with MS in a sociodemographically diverse population from the United States, and to estimate the associations between health literacy and health behaviors, comorbidities, and health care utilization. We hypothesized that lower health literacy would be associated with a higher frequency of smoking, obesity, and greater health care utilization.The North American Research Committee on Multiple Sclerosis (NARCOMS) Registry is a voluntary self-report registry for people with MS, developed by the Consortium of MS Centers . We haver=.71, P<.001)  vs ≥4 [adequate literacy]). First we evaluated the association between participant characteristics and having functional/adequate literacy. We constructed separate models for the METER and the NVS. Second, we separately modeled the association of health literacy with the outcomes of any ER visits and any hospitalizations. The independent variables considered for each regression model are described below.For gender, female was the reference category. Race was categorized as white (reference group), and nonwhite. Education was included as indicator variables for high school diploma or less (reference group), Associate’s Degree or Technical Degree, Bachelor’s Degree, and post-graduate degree. Annual household income was included as indicator variables for <$15,000 (reference group), $15,000-29,999, $30,000-49,999, $50,000-100,000, and >$100,000, or declined to answer. Insurance status was included as indicator variables for private, public only (reference group), or none. Age was categorized as ≤35 (reference group), >35 to ≤50, >50 to ≤65, and >65 years. Using PDDS, participants were classified as having mild (0-2), moderate (3-4), or severe  disability [Assumptions of models were tested using standard methods . For eacP<.001), to have a lower level of education (P<.001) and lower annual income (P=.007). They did not differ with respect to gender. Nonresponders were slightly younger  than responders . Mean age at onset of MS symptoms was also younger in nonresponders  than responders , but the difference of less than a year is unlikely to be clinically relevant. Of those who completed the questionnaire, 8934 (99.06%) were US residents and were included in this analysis. The demographic and clinical characteristics of the responders are summarized in Of 13,020 eligible participants, 9019 (69.27%) completed the spring 2012 questionnaire. As compared to responders, nonresponders were more likely to be nonwhite .The internal consistency reliability of the NVS was .74, and of the eHEALS was .94. Most respondents performed well on the health literacy instruments. On the METER, 1.84% (160/8719) had low literacy, 17.12% (1493/8719) had marginal literacy, and 81.04% (7066/8719) had functional literacy. On the NVS, 10.81% (966/8933) of respondents had a high likelihood of inadequate literacy, 14.56% (2267/8933) possibly had inadequate literacy, while 74.62% (6666/8933) had adequate literacy. Only 1.03% (90/8718) of participants had low literacy on both the METER and the NVS, while 65.52% (5712/8718) had functional literacy/numeracy on both instruments. METER scores correlated weakly with NVS scores  and the NVS . The mean (SD) eHEALS score was lower among persons with low literacy on the METER  than among persons with functional literacy . Similarly, the mean (SD) eHEALS score was lower among persons with a high likelihood of inadequate literacy on the NVS  than among persons with adequate literacy .The mean (SD) score on the eHEALS was 28.15 (18.57). eHEALS scores correlated quite weakly with scores on the METER , higher levels of education, higher levels of income, and older age, were associated with higher odds of having functional health literacy as assessed by the METER . Higher In total, 6973 (78.05%) of 8934 participants reported one or more comorbid conditions with 2868 (32.1%) reporting hypertension, 3328 (37.25%) depression, 2804 (31.39%) hyperlipidemia, 1413 (15.82%) migraine, 1152 (12.89%) autoimmune thyroid disease, and 1032 (11.55%) reporting cancer. The remaining comorbidities were reported by fewer than 10% of respondents. Most respondents did not smoke currently  and 3061/8797 (34.78%) denied any alcohol consumption. The mean (SD) BMI of respondents was 26.93 (6.48), with 2625/8741 (30.03%) being overweight and 2239/8741 (25.61%) being obese.P=.07 for linear trend) and on the NVS . Respondents who reported being nonsmokers were more likely to have functional literacy on the METER and adequate literacy on the NVS than smokers .The proportion of respondents with any comorbidity was slightly higher among those with greater health literacy on the METER  or hospitalization (P<.001) if they had higher literacy on the METER  respondents presented to an emergency room and 831/8792 (9.45%) were hospitalized. Participants were less likely to report an ER visit . In a multivariable logistic regression model adjusting for income, disability, and cognitive impairment, low literacy on the METER was associated with 13% increased odds of any ER visit  .In an unadjusted logistic regression model, low literacy on the NVS was associated with 58% increased odds of any ER visit . In a multivariable model adjusting for income, disability, and cognitive impairment, low literacy on the NVS was still associated with increased odds of any ER visit  .In an unadjusted logistic regression model, low literacy on the METER was associated with 28% increased odds of any overnight hospitalization . In a multivariable logistic regression model adjusting for gender, income, disability and cognitive impairment, low literacy on the METER was associated with 19% increased odds of any hospitalization  .In an unadjusted logistic regression model, low literacy on the NVS was associated with 58% increased odds of any overnight hospitalization . In a multivariable logistic regression model, low literacy on the NVS was associated with 17% increased odds of any hospitalization  .We investigated health literacy in a sociodemographically diverse population of persons with MS. We found that 65.52% of respondents had functional health literacy on both the METER and the NVS. Furthermore, functional literacy was associated with greater comfort and perceived skill at using electronic health information as assessed using eHEALS. Although most respondents performed well on the METER and the NVS instruments, lower health literacy was associated with an increased risk of smoking, overweight and obesity, comorbidity, visits to the emergency room and overnight hospitalizations.As assessed by the METER, 81% of the NARCOMS population had functional literacy while nearly 75% had adequate literacy as assessed by the NVS; the latter instrument has a greater emphasis on numeracy. We were unable to identify other studies which have evaluated this issue in MS. Findings varied in other chronic diseases. Approximately 70% of individuals diagnosed with COPD have functional health literacy , as compSociodemographic characteristics associated with greater odds of having functional health literacy included females, older age, higher socioeconomic status, normal self-reported cognition, and lower levels of disability. The association of socioeconomic status and health literacy is consistent across populations, including those with heart failure , chronicLower health literacy was associated with a greater frequency of smoking and obesity, but a lower frequency of regular alcohol use and comorbidity. Findings in other populations regarding these associations have been inconsistent . Our finThe response rate was 69.27% and responders were more likely to be white and to have a higher annual income; thus, our findings may not be applicable to nonwhites and those of lower socioeconomic status. The NARCOMS population does not fully represent the MS population in the United States, but its characteristics are similar to those reported for other MS populations ,35. FurtHealth literacy is under-studied in MS. Our findings suggest that it is associated with adverse health behaviors, and increased health care utilization. Future work should seek to develop better methods of defining and assessing health literacy in MS population, confirm these findings, elucidate causal pathways, examine a broader range of health outcomes including adherence to therapy, and ultimately, evaluate the impact of interventions aimed at improving health literacy in the MS population.