This paper reports the process of establishing a transparent, accountable, evidence-based program for introduction of new technologies and clinical practices (TCPs) in a large Australian healthcare network. Many countries have robust evidence-based processes for assessment of new TCPs at national level. However many decisions are made by local health services where the resources and expertise to undertake health technology assessment (HTA) are limited and a lack of structure, process and transparency has been reported.An evidence-based model for process change was used to establish the program. Evidence from research and local data, experience of health service staff and consumer perspectives were incorporated at each of four steps: identifying the need for change, developing a proposal, implementation and evaluation. Checklists assessing characteristics of success, factors for sustainability and barriers and enablers were applied and implementation strategies were based on these findings. Quantitative and qualitative methods were used for process and outcome evaluation. An action research approach underpinned ongoing refinement to systems, processes and resources.A Best Practice Guide developed from the literature and stakeholder consultation identified seven program components: Governance, Decision-Making, Application Process, Monitoring and Reporting, Resources, Administration, and Evaluation and Quality Improvement. The aims of transparency and accountability were achieved. The processes are explicit, decisions published, outcomes recorded and activities reported. The aim of ascertaining rigorous evidence-based information for decision-making was not achieved in all cases. Applicants proposing new TCPs provided the evidence from research literature and local data however the information was often incorrect or inadequate, overestimating benefits and underestimating costs. Due to these limitations the initial application process was replaced by an Expression of Interest from applicants followed by a rigorous HTA by independent in-house experts.The program is generalisable to most health care organisations. With one exception, the components would be achievable with minimal additional resources; the lack of skills and resources required for HTA will limit effective application in many settings. A toolkit containing details of the processes and sample materials is provided to facilitate replication or local adaptation by those wishing to establish a similar program.The online version of this article (doi:10.1186/s12913-015-1178-4) contains supplementary material, which is available to authorized users. New health technologies and clinical practices (TCPs) are defined as therapeutic interventions or diagnostic procedures that are considered by a reasonable body of clinical opinion to be significantly different from existing clinical practice. Therapeutic interventions include prostheses, implantable devices, vaccines, pharmaceuticals and medical, surgical or other clinical procedures .Australia has robust evidence-based processes for assessment of new health technologies, clinical practices and medications through the national Medical Services Advisory Committee (MSAC) and Pharmaceutical Benefits Advisory Committee (PBAC). Although these processes are rigorous and provide trustworthy information, they do not address all the requirements of healthcare decision-makers. MSAC decisions only cover therapeutic and diagnostic procedures provided by doctors; they do not include activities of nursing and allied health professionals, models of care or service delivery. PBAC decisions only consider pharmaceuticals for community use and do not include some therapeutic agents used solely in the hospital context. Not all topics being considered by decision-makers have been addressed in national recommendations and central agencies cannot take into account local factors such as population needs, organisational priorities, budgets, capacity or capability. Hence many decisions about the use of TCPs have to be made at the state, regional and hospital levels. At national level, evidence-based assessment and development of recommendations for application and funding of new TCPs is enabled by rigorous processes, underpinned by appropriate resources and expertise. However at the local level limitations in processes, resources and expertise means that decision-making is undertaken with varying degrees of rigour, structure and transparency –4.Monash Health  is a large health service network providing primary and secondary care in the south east of Melbourne and tertiary and quaternary care in specialist areas across the state of Victoria, Australia. In 2000, Monash Health established the first Technology/Clinical Practice Committee (TCPC) in Victoria to assess TCPs prior to their introduction. Since then, a number of factors have influenced health agencies around the world in how they approach assessment of new TCPs. These global challenges arise from rapid advances in health technologies; consumers’ desires to be well informed and participate in decision-making; imperatives for transparent, accountable and evidence-based decision-making (EBDM); and the need to get best value from finite or decreasing resources.Although early leaders in this area, the TCPC acknowledged that there were opportunities for improvement. Limitations of the Monash Health system included inadequate transparency and lack of explicit criteria in decision-making, lack of high quality information for decision-making, meetings called at short notice resulting in lack of representative views and inadequate preparation time, lack of awareness of requirements for applications, need for improved accessibility of application materials and limited resources to monitor newly introduced TCPs. The need for more rigorous processes to ensure safe introduction of new TCPs was identified as a priority for the organisation.This project was undertaken by the Centre for Clinical Effectiveness (CCE), an in-house ‘Evidence Based Practice Hospital Support Unit’ providing expertise in evidence synthesis, implementation and evaluation .The aim of the project was to establish a sustainable, transparent, accountable and evidence-based program for introduction of new TCPs in the local healthcare setting.This paper aims to outline the development, implementation and evaluation of the program.A toolkit for introduction of new TCPs in hospitals and health care organisations has also been developed which aims to assist health service staff to establish similar programs by providing detailed descriptions of the components, templates of useful documents and links to resources [Additional file What is best practice for introduction of new TCPs?How can best practice be implemented most effectively?What are the outcomes of implementation and the factors for success?This project was undertaken using the SEAchange model for sustainable, effective and appropriate change in health services . The modMixed methods were used for process and outcome evaluation. Quantitative methods included audit, surveys and document analysis. Qualitative methods included workshops, individual and group discussions and feedback forms.An action research approach was adopted based on the ‘researcher as facilitator for change’ model defined by Meyer; researchers working explicitly with and for people rather than undertaking research on them , 8. In tA timeline for the project is included in the toolkit [Additional file ‘Decision-makers’ were members of the TCPC including an Executive Sponsor; representatives with expertise in operations, finance, evidence-based practice, ethical and legal considerations; clinical program directors and health service consumers. Further details are available in the Terms of Reference [Additional file ‘Administrators’ were health service staff managing the processes related to making, implementing, monitoring and reporting decisions related to introduction of new TCPs and specialist staff who provided expertise to assist applicants in use of evidence (CCE) and health service utilisation data , coding , credentialing and scope of practice  and development of business cases (Finance and Business Managers).‘Applicants’ were clinicians  or clinical managers who were seeking authorisation to introduce a new TCP.Three stakeholder groups participated in the design, implementation and evaluation of the TCP program.Data were collected in an ongoing process over two years and methods were designed to minimise the time and effort required of participants. Scheduled meetings of the TCPC were used for formal workshops and informal group discussions with the decision-makers. Informal interviews with administrators were undertaken during routine meetings or by appointment. The applicants were clinicians based across a number of campuses who found it difficult to attend additional meetings, hence a range of options for individual feedback was provided  AND (committee OR guide OR policy OR procedure) in the Google Advanced Search function.Critical appraisal relevant to the study design was planned, however no research studies were identified and the expert guidance documents ascertained contained no methods to allow critical appraisal.The principles identified in the literature and local needs assessment were collated and tabulated into a Best Practice Guide. Program components were developed through stakeholder consultation and feedback.A checklist developed for previous CCE projects was used to assess the likelihood of success and sustainability of the proposed changes. The characteristics of success were derived from the work of Grol and Grimshaw; Grol, Wensing and Eccles; Greenhalgh et al. –11 and tBarriers and enablers to the proposed changes were identified by decision-makers and administrators in group and individual discussions, and by applicants in individual discussions, feedback forms at the end of all documents and email invitations to provide input. The project team used the checklist for success and sustainability and the classification of barriers and enablers by Grol and Wensing  as prompA formal evaluation plan was developed. Full details including the evaluation questions for each component, indicators, methods, sources and timing of data collection, and reporting schedule are available in the toolkit [Additional file Current practice was mapped against the identified principles in the Best Practice Guide to provide baseline data. Planned evaluations were undertaken at 12 and 24 months [Additional file Following the initial evaluation period, the TCPC and Secretariat continued to collect, analyse and act on feedback as a quality improvement activity.The Monash Health Human Research and Ethics Committee approved this project as a Quality Assurance activity (Application Number 09195Q).Twenty-five needs were identified Table . These cFive relevant publications from national and state bodies were identified , 14–17. The publications identified standards, rules, criteria or principles that they recommended for TCP programs within hospitals or health care organisations. For the purposes of this paper, these are referred to collectively as ‘principles’ for good practice in introduction of TCPs.Twenty-seven principles were extracted. There was considerable variation in content between the documents with only six principles common to all five publications : effect of state and national activities related to TCPsEconomic and political context (n = 10): lack of time and resources, lack of awareness of current decision-making structuresOrganisational context (n = 5): lack of influence, effect of perceptions and power relationshipsSocial context (n = 4): need for adequate consumer representation in decision-making, limitations in quality of patient information brochuresPatient (n = 17): lack of awareness, lack of knowledge and skills, poor complianceIndividual professional (n = 6): complexity, time requirement, perceived lack of advantageInnovation  of the publications. These focused on overarching issues relating to governance, use of evidence in decision-making, and application and monitoring processes.Fifty-eight principles were cited by less than one third (5 or less) of the guidance documents. These fell into two groups: those that specify more detail in the governance, decision-making and application processes and those that address aspects of reporting, administration, provision of resources and support, and evaluation of the program. Only five publications referred to consumer representation and only one suggested reporting outcomes to local consumer health councils or networks . Only onThe 10 local needs were reframed as principles; seven of which were duplicated in the literature. When added to the 109 published principles these brought the total to 112.The initial literature review did not identify any recommendations for provision of resources, although Monash Health staff had considered this important. The principles related to resource provision from the first needs analysis were now included in the literature, however none of the documents noted principles identified by Monash Health staff for adequate allocation of staff with appropriate skills to manage and support the decision-making process or the need for evaluation and improvement of the systems and processes for introducing new TCPs.Monash Health met 89 out of 112 principles. Most of the unmet principles related to the need for objective assessments undertaken by independent experts and direct ‘head to head’ comparisons of the TCPs under consideration.The Best Practice Guide was revised to include the additional principles and categorise them into the seven program components. In addition to those above, Monash Health staff had identified and implemented many other principles during the implementation phase, 11 of which were not found in the literature. The recommendation for an Expression of Interest to replace the current application process was not included as it was considered to be a local solution rather than a universal principle for best practice. The final total was 122 principles for good practice (Additional file The unmet principles were considered and amendments to the program were proposed. Membership of the TCPC was changed to increase the seniority, credibility and influence of the committee to make and implement appropriate and acceptable decisions. The other amendments focused on improving the quality of information provided to decision-makers, predominantly through the application process.In the new model the previous lack of objectivity, time and skills is addressed by providing resources so that independent experts can undertake the work. To minimise unnecessary resource use, and in contrast to the Canadian and New Zealand models, the information is requested in stages in the Monash Health program, each stage predicated on a positive decision at the stage before Fig. .Fig. 3ReApplicants submit an Expression of Interest in a much briefer document than the previous application form which greatly reduces their time commitment [Additional file Considering the problems inherent in an ‘applicant-driven’ model and the successes of the international models using independent experts, Monash Health anticipates that use of organisational resources to provide better information to underpin decisions will be cost-effective.This process is currently being piloted and refined.A systematic approach was used to ascertain evidence to underpin the new TCP program; however no research was identified for organisational decision-making and clinical governance. Although the documents available were from credible organisations and considered to be ‘expert advice’, no quality appraisal could be undertaken to validate the recommendations. This is not likely to have a significant impact on the process or outcomes of this initiative as the recommendations reflect good practice principles that have been well established elsewhere eg transparent, evidence-based, consumer participation, etc.The aims of transparency and accountability in decision-making were achieved. The process and requirements are explicit, decisions are published on the internet, outcomes are recorded and activities are reported internally and externally.Use of an evidence-based approach to change guided by information from the literature, local experts and consumersMultidisciplinary stakeholder involvement, including consumers, in development, implementation and evaluation of the project and representation in the systems and processes of the ongoing programAssessment of barriers and enablers, characteristics of success and factors for sustainability followed by tailoring of strategies to maximise the benefits and minimise the problems identifiedImplementation in a long-term ‘piloting’ mode that captured and acted upon user feedback for continuous improvementCredibility of a program underpinned by international best practiceProvision of sufficient resources to undertake the project and deliver the programCommitment, support and leadership from the Board, Executive, Senior Management and Clinical DirectorsSkills of the CCE team in Evidence-Based Practice, knowledge brokerage and implementation of changeAlmost all components of the TCP program were implemented effectively. A range of factors are likely to be responsible for this success.The aim of sustainability was also met, at least for the foreseeable future while Monash Health ensures the relevant skills and resources are available to provide high-quality information for decision-making.It is not uncommon in health services for new initiatives to fail if contextual factors are changed. At the end of the establishment phase the CCE project team handed over to the Medical Governance Office as this was thought to be more suitable and sustainable for ongoing administration of the committee. A second handover to the Director of Medical Services was undertaken and then CCE took over the role again at the beginning of the revision phase. All three handovers went smoothly. The systems, processes, documents and resources proved to be readily transferrable and the program ran seamlessly throughout the moves between departments.Monash Health demonstrated effective leadership in this area. The high standards achieved by this initiative were acknowledged through a national award and multiple requests to assist decision-makers in other contexts with translation to their settings. Many health services have yet to address the issues related to introduction of new TCPs and it is still common for others to rely on processes that are not evidence-based, transparent or accountable .Minor problems were amended as they arose; these are captured in the needs assessments and barrier analyses and the strategies to address them are summarised in the toolkit. The main areas of concern related to non-compliance with the application requirements which resulted in incorrect or inadequate information being provided for decision-making. This is being addressed in the revision phase.The Australian government has called for reform of post-market surveillance of health technologies to strengthen patient safety and value for money for taxpayers and an international policy forum has proposed that a minimum dataset be developed to focus monitoring activities , 35. ThiIntroduction of mechanisms for prioritisation would improve the program . The curThe structure of the program could be improved by introducing an eighth component for ‘Implementation’. The current Application and Monitoring components include principles that focus on the safety of implementation of a new TCP. Having a specific component would not only capture these but could also include principles that focus on, and highlight the importance of, an evidence-based approach to the implementation process.The current system is reactive; it responds to individual applications which are driven by non-systematic factors such as clinician’s interests and exposure to promotion of new TCPs. This could be improved through a systematic proactive approach where the organisation seeks out information on new TCPs that are already proven to be safer, more effective or most cost-effective than current practice and considers their fit with organisational objectives and the opportunity costs and risks incurred if they are introduced or not.There is considerable waste of resources when each health service replicates the information gathering steps to make the same decisions. Sharing of information could reduce this duplication. Monash Health publishes its Decision Summaries on the internet but provision of a central website to house this information at a state or national level might encourage similar publications by others and facilitate access and utilisation for decision-making.The initial program model was sustainable but proved inadequate to address the aim of robust EBDM. The revised model is likely to meet the aim but, due to the resources required to deliver the evidence, may not be sustainable. Local decisions need to consider local factors however a systematic review of the literature should not be duplicated in each health service. Methods to encourage and facilitate publication of systematic reviews conducted for local decision-making could be explored.The lack of research into decision-making processes for introduction of new technologies at the local level and the limitations of ‘knowledge purveyors’ in this context have been noted , 37. ThiAlthough it is recommended in the Best Practice Guide (Additional file The Technology/Clinical Practice Program was established using an evidence-based approach to development, implementation and evaluation. The program components were based on a review of the literature, consultation with experts and stakeholders, assessment of characteristics of successful change models and factors for sustainability, identification of barriers and enablers to introduction of best practice and experience from implementation and evaluation in a large health service network.A toolkit containing details of the processes and resources for implementation is provided to facilitate replication or local adaptation by those wishing to establish a similar program. The components are likely to be generalisable to most health care organisations and, with the exception of the systematic review process, would be achievable with minimal additional resources.Expertise for effective HTA is not available in most health services, but even if it were, duplication of systematic reviews for the same TCP would be a waste of very limited resources. Some duplication is required for assessment of local factors such as capacity, capability and access but methods to share information that is applicable to all need to be explored.Further research is required into EBDM for resource allocation at local level.


There is evidence that in Europe and North America many species of pollinators are in decline, both in abundance and distribution. Although there is a long list of potential causes of this decline, there is concern that neonicotinoid insecticides, in particular through their use as seed treatments are, at least in part, responsible. This paper describes a project that set out to summarize the natural science evidence base relevant to neonicotinoid insecticides and insect pollinators in as policy-neutral terms as possible. A series of evidence statements are listed and categorized according to the nature of the underlying information. The evidence summary forms the appendix to this paper and an annotated bibliography is provided in the electronic supplementary material. Since their introduction in the 1990s, their use has expanded so that today they comprise about 30% by value of the global insecticide market  a strong evidence base involving experimental studies or field data collection, with appropriate detailed statistical or other quantitative analysis;— [Dxp_op] a consensus of expert opinion extrapolating results from related ecological systems and well-established ecological principles;— [Eupp_ev] some supporting evidence but further work would improve the evidence base substantially; and— [Srojns] projections based on the available evidence for which substantial uncertainty often exists that could affect outcomes.— [PThe literature on pollinators and neonicotinoids was reviewed and a first draft evidence summary produced by a subset of the authors. At a workshop, all authors met to discuss the different evidence components and to assign to each a description of the nature of the evidence using a restricted set of terms. We considered several options to describe the nature of the evidence we summarize including the GRADE  system wThese categories are explicitly not in rank order.A revised evidence summary was produced and further debated electronically to produce a consensus draft. This was sent out to 34 stakeholders or stakeholder groups including scientists involved in pollinator research, representatives of the farming and agrochemical industries, non-governmental organizations concerned with the environment and conservation, and UK government departments and statutory bodies responsible for pollinator policy. The document was revised in the light of much helpful feedback. Though many groups were consulted, the project was conducted completely independently of any stakeholder and was funded by the Oxford Martin School (part of the University of Oxford).3.The summary of the natural science evidence base concerning neonicotinoid insecticides and insect pollinators is given in appendix A, with an annotated bibliography provided as the electronic supplementary material.4.The purpose of this project is not to conclude whether neonicotinoids are ‘safe’ or ‘dangerous’ but to try to help set out the existing evidence base. When neonicotinoids are used as seed dressing on crops visited by pollinators there is no doubt that these systemic insecticides are typically present in pollen and nectar and so bees and other pollinators can be exposed to them ,8. The chttp://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:093:0001:0084:EN:PDF and http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:093:0085:0152:EN:PDF). These include substantial evidence on toxicity to non-target organisms (including honeybees) and a range of further studies that will, in some cases, escalate to full-scale field trials of toxicity. The data generated in such studies are not typically in the public domain, or only in a form summarized by the regulatory agencies, and hence we have not been able to include reference to them. There are understandable commercial reasons for the withholding of this information, though the chief reason is not that it contains proprietary intellectual property but that the information would be commercially advantageous to a competitor in registering the compound when it is out of licence. We wonder if registration rules might be amended to allow this type of data to be published, a clear public good, without disadvantaging companies that had invested in its collection.For this topic, the published literature is a small fraction of the evidence that has been collected. The process of registering a new insecticide requires the production of detailed environmental risk assessments .There is clear evidence of the great value of neonicotinoids in agriculture  as well


Transnational tobacco companies (TTCs) submitted evidence to the 2012 UK Consultation on standardised packaging (SP) to argue the policy will have detrimental economic impacts and increase illicit tobacco trade.A content analysis of the four TTC submissions to the consultation assessed the relevance and quality of evidence TTCs cited to support their arguments. Investigative research was used to determine whether the cited evidence was industry connected. Fisher's exact tests were used to compare the relevance and quality of industry-connected and independent from the industry evidence. The extent to which TTCs disclosed financial conflicts of interest (COI) when citing evidence was examined.We obtained 74 pieces of TTC-cited evidence. The quality of the evidence was poor. TTCs cited no independent, peer-reviewed evidence that supported their arguments. Nearly half of the evidence was industry-connected . None of this industry-connected evidence was published in peer-reviewed journals (0/35) and 66% (23/35) of it was opinion only. Industry-connected evidence was of significantly poorer quality than independent evidence (p<0.001). COIs were not disclosed by TTCs in 91% (32/35) of cases.In the absence of peer-reviewed research to support their arguments, TTCs relied on evidence they commissioned and the opinions of TTC-connected third-parties. Such connections were not disclosed by TTCs when citing this evidence and were time consuming to uncover. In line with Article 5.3 of the Framework Convention on Tobacco Control and broader transparency initiatives, TTCs should be required to disclose their funding of all third-parties and any COIs when citing evidence. Transnational tobacco companies (TTCs) have vehemently opposed proposals for the introduction of standardised packaging (SP) for tobacco products in the UK,18Such economic arguments can be powerfully used under the framework of Better Regulation (BR), introduced from the mid-1990s across jurisdictions globally, including the EU and UK. Better, or smart, regulation aims to improve policy-making by increasing the role of evidence in policy-making as well as formalising opportunities for affected interests to be consulted.25In light of the above, TTCs’ long-standing history of manipulating evidence in their own interest36–39The situation is further complicated by TTCs’ growing use of the third-party technique: using a seemingly independent messenger with a better reputation and greater credibility to convey arguments.18This paper therefore aims to assess the quality and relevance of the evidence that TTCs cited in their submissions to the UK's 2012 consultation on SP to argue that the policy would lead to negative economic impacts and an increase in the illicit tobacco trade. It also examines the degree to which TTCs cite ‘evidence’ from those with TTC connections  and how transparent TTCs are about these connections in their written submissions. The UK Intellectual Property Office's guidance on standards of evidence used in the development of policy state: “An important part of public policy-making is transparency. It should be made clear who has commissioned and funded the research as well as who has carried it out.”45Consultation submissions were obtained, via company websites, for the four TTCs operating in the UK—Imperial Tobacco, Japan Tobacco International (JTI), BAT and PMI. Every citation (hereafter referred to as evidence) made in each TTC submission to support arguments on alleged illicit trade and economic impacts of SP was recorded. In line with questions asked in the Department of Health's Impact Assessment,Unlike our previous work, where we recorded only formal written evidence cited,Copies of evidence cited by TTCs were obtained and for each, the author, title, date and source were recorded in an Excel spreadsheet. The quality of the evidence and its relevance to SP was assessed using criteria based on existing literature in this field.Evidential quality was assessed using three criteria: independence from the tobacco industry, nature of the evidence and publication route .Independence: For each piece of evidence cited, the TTC's submission was searched for a funding or a conflict of interest (COI) statement. If no disclosure was found, we noted that the TTC had not declared any financial relationship between themselves and the authors of the cited evidence. At this stage, to clarify whether the evidence was either connected or independent of industry, each piece of evidence was searched for a funding or COI statement. If none was present within the evidence document itself then a further series of checks was performed , statistics or facts with no opinion expressed , opinion , or strategy documents  and tobacco packaging, standardised or otherwise. Evidence about illicit trade or economic issues but not packaging was coded as ‘less relevant’ to SP regulation.8To determine its relevance to SP policy, the Evidence was coded by two researchers (KAER and JH) with 94% concordance. All discrepancies were resolved.Following categorisation of the evidence, we compared the relevance and quality of the evidence connected to and independent of industry using two-tailed Fisher's exact tests. The measures of quality used for this analysis were the nature of the evidence and publication route. For nature of the evidence we combined research, publication of facts and statistics, and strategy documents and compared these with opinion. For publication route, we combined articles published in peer-reviewed journals or other academic outlets along with official government and parliamentary publications and compared them with those published by private organisations or in the press.Finally, for TTC-cited evidence coded as industry-connected, the number of instances where TTCs had disclosed this link in their submissions was recorded.Box 1Negative economic consequences“The Tobacco Retailers Alliance UK, a coalition of 26 000 independent shopkeepers who sell tobacco products, has warned its members that: ‘…banning cigarette branding would directly threaten small shops. For many of you, tobacco sales make up around a third of turnover, sometimes more. . . . During the busy times, such as the morning rush, there is a real risk that customers who have to wait will go to supermarkets and larger shops, which have more staff and therefore shorter transaction times.’”2In its submission JTI does not disclose that it commissioned the Chaudhry and Zimmerman report.71“Experts from law enforcement officers to academics have concluded that plain packaging will increase demand for illicit tobacco products. For example, plain packaging will likely cause an increase in the black market for smuggled branded tobacco according to nearly 70% of current UK police officers who responded to a recent survey” 1In its submission PMI does not disclose that it commissioned the police officers survey136Imperial, JTI, BAT and PMI collectively cited 83 pieces of unique evidence to support illicit trade and economic arguments against SP. We were able to obtain 74; 11 were predominantly about economic issues, 50 were about illicit issues, while 13 pieces of evidence referred to both. Examples of these arguments and the evidence used to support them are given in Overall, the quality of the cited evidence was very poor; this was particularly the case for industry-connected evidence .Almost half of the evidence cited  was financially connected with the tobacco industry, while 51% (38/74) was independent of the tobacco industry. For one piece of evidence it could not be determined either way.Nearly half of the evidence consisted of opinion only  of the evidence was published in either a peer-reviewed journal (2) or other academic outlet (3), while 46% (34/74) was privately published, 32% (24/74) came from official government or parliamentary sources and 15% (11/74) was published in the press . The pubOf the 74 pieces of evidence obtained 42/74 (57%) were relevant to either SP or packaging in general, 31/74 (42%) about tobacco but not packaging and 1/74 (1%) was unrelated to either packaging or tobacco. The odds of evidence being about packaging were 34 times higher if it was industry-connected (32/35) compared to independent evidence  pieces of evidence. The authors of this linked evidence were either TTC employed, created, commissioned, part-funded , involved in the tobacco supply chain or in receipt of TTC hospitality .Tobacco Manufacturers’ Association—the TTCs’ trade association in the UKTobacco Retailers Alliance,created and entirely financed by the Tobacco Manufacturers’ Association.British Brands Group and the Anti-Counterfeiting Group,created by one or more TTCs.British Brands Group, neither Imperial Tobacco, BAT nor PMI disclosed that they were members. However, it must be noted that JTI, which is not a member, did disclose in its submission that although it was not a member, other tobacco companies were.Only in two instances , which was not declared.International Chamber of Commerce (ICC)ICC Business Action to Stop Counterfeiting and Piracy.UNITE, TTCs cited seven research reports they had supply chain companies, two from packaging companies (who list TTCs as clients)European Carton Makers Association,Four pieces of evidence were authored by hospitality from JTI.An open letter cosigned by 51 Members of Parliament to Health Secretary Andrew LansleyErik Bloomquist, an equity analyst at Berenberg Bank, was cited.Common Sense Alliance, an antiregulatory lobby group that receives financial support from BAT.136Finally, in three cases, the authors’ links to the tobacco industry were more obscure. A report by The evidence TTCs used to support their argument that SP would have adverse economic and illicit trade impacts was of very low quality. TTCs did not cite any independent, peer-reviewed evidence that supported these arguments. Instead they relied on research they had commissionedWe set the parameters of this study to focus exclusively on the evidence TTCs used to support illicit trade and economic arguments (as defined in the Impact Assessment)9As we were interested specifically in the quality of the evidence used by TTCs, we only counted unique pieces of evidence, although many were cited by more than one company and sometimes by all four. Finally, we were unable to identify industry links to three of the business organisations who co-authored, with other TTC-connected business organisations, opinion statements cited by the TTCs  or misrepresent the nature of the trade in order to down-play the extent of tobacco industry involvement.127Taken together, the absence of independent evidence, or even independent opinion, that SP will increase illicit trade revealed in this paper, the questionable veracity of industry-funded data, the fact that the industry's previous illicit trade predictions have not materialised and PMI's explicit intention to utilise the illicit trade argument to “ensure that PP [plain packaging] is not adopted in the UK”,The same is true of TTCs’ claims that SP will have adverse economic consequences. Independent research has found no evidence to support industry predictions that small businesses in Australia would suffer from customers moving from small convenience stores to bigger supermarkets.39TTCs extensive use of third-party voices as authors of the ‘evidence’ creates problems for policymakers trying to assess the reliability of evidence. Tobacco companies’ failure to declare these COI in their 2012 consultation submissions demonstrates that, despite decades of criticism and a requirement for transparency in Article 5.3, TTCs continue to rely heavily on the third-party technique to disseminate their messages via those with more credible voices than their own. Furthermore, the number and nature of organisations and individuals cited creates the impression of a large, expert and seemingly unconnected cohort, all opposed to or concerned about SP. PMI's leaked SP-opposition strategy named many  of the individuals and organisations cited in the TTC submissions (in the context covered in this paper) as ‘influencers’ or ‘media messengers’  are granted an explicit role to provide new evidence and to contest existing evidence and data. However, inadequate COI disclosure requirements mean that BR fails to take account of TTCs’ longstanding history of manipulating evidence to serve their own interests. In this instance, the BR framework has enabled TTCs to more effectively frame and use their misleading arguments. This has contributed to uncertainty about policy impacts and consequent regulatory delays.131If TTCs are allowed to continue to submit evidence to public consultations as part of BR, it should be compulsory for them, in line with 5.3, to exercise transparency by disclosing any potential COI, not only of their own position, but also that of the evidence they use to build their case either for or against a policy proposal. Such disclosures are now standard practice when submitting research to health-related academic journals.Given the findings of this paper, the requirements of Article 5.3 and the overwhelming evidence of the tobacco industry's misuse of evidence, the terms of their inclusion in stakeholder consultation processes require some reassessment. Our analysis reveals that transnational tobacco companies (TTCs) have cited no independent, peer-reviewed evidence that supports their case. Instead they relied on evidence they had commissioned and the opinions of TTC-connected third-parties, providing further evidence that TTCs exaggerate the threat of illicit tobacco and the negative economic consequences of policy as part of a deliberate strategy to ward off regulation.TTCs have not disclosed relevant conflicts of interest within their submissions, highlighting incompatibility between Better Regulation and Article 5.3 of the Framework Convention on Tobacco Control.


Recent World Health Organization (WHO) guidelines not only advise countries on what treatment to give patients and when, but also how to roll this out in countries. Priya Shetty reports. Just a decade ago few patients in developing countries were receiving antiretroviral (ARV) medicines; HIV infection was in most cases a death sentence. In 2003, the 3 by 5 campaign was launched to extend this life-saving treatment to those in need.With nearly 10 million HIV-positive people receiving treatment in developing countries at the end of 2012, the global roll-out of ARVs has been a major public health success – one in which clinical guidelines play an important role. In South Africa, for example, adult life expectancy has increased by 11 years since 2004, when treatment was first provided by the public health system.But, over the years, HIV programme managers started to feel overwhelmed. “The message we were getting was that WHO was generating so many guidelines, technical updates, programmatic updates and so on that it was quite challenging to implement them,” says Dr Philippa Easterbrook from WHO’s HIV department.“The idea was to create a one-stop shop,” says Easterbrook, “which would unite existing advice with new guidance to provide a complete view of all the current information on HIV treatment.” So far, the feedback from countries has been positive, she says, and as a result, a similar approach is being pursued with other HIV department guidelines, including plans for consolidated guidelines on: strategic information, HIV testing and hepatitis C screening, care and treatment.“The idea was to create a one-stop shop.”Philippa EasterbrookConsolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach, including a new recommendation to initiate treatment at an earlier stage of the infection – thus increasing the number of people globally who would be eligible for treatment. In 2013, WHO issued the The WHO recommendation change was based on evidence accumulated over the years showing that early initiation of treatment allows people with HIV infection to live longer, healthier lives while reducing the risk of infecting others. And it was estimated that such a change could avert an additional 3 million deaths and prevent 3.5 million more new HIV infections by 2025. But increasing the pool of people who are eligible for ARV treatment poses a major challenge for countries, like South Africa, facing health worker shortages and other health systems problems.Despite that, Dr Yogan Pillay, Deputy Director General, HIV/AIDS, Tuberculosis and Maternal Child and Women’s Health at the National Department of Health in South Africa, says the consolidated HIV guidelines were “very well received” not least because these were accompanied by advice on how to deliver these services and, of course, the ultimate goal is to save even more lives. “The regional workshops on service delivery that were held after the guidelines were launched were very useful,” Pillay says.The global health landscape has changed considerably since WHO was founded in 1948 and started issuing guidance on a wide range of health areas to its Member States. But, while its mandate to “make recommendations with respect to international health matters” is anchored in its Constitution, the process for developing recommendations has evolved over the years.The most recent shift in WHO’s approach to making recommendations came after an external review of the guideline development process in 2007. The review concluded that WHO guidelines drew too heavily on expert opinion, and not enough on systematic reviews of evidence, and that there was a lack of transparency about the way they were developed. There was also a question over the extent to which they were driven by public health need, says Dr Charles Penn, who is currently the chair of the WHO Guidelines Review Committee that was set up in 2008. Since then, WHO has put in place a set of standards for developing guidelines, which are constantly evolving as methods in the field change, to ensure that they reflect the best available scientific evidence, a shift that the Guidelines Review Committee oversees.A key feature of this new emphasis on systematic use of evidence has been the adoption of the grading of recommendations, assessment, development and evaluations methods, best known by its acronym GRADE, to assess the quality of evidence and strength of recommendations based on this evidence.The idea, says Penn, is to separate evidence retrieval from expert analysis, “so that expert panels are looking at the evidence put in front of them, and not coming with their own preconceptions and biases”.But the quality of evidence is not the same in every field, which is a challenge for those intent on producing strong, evidence-based guidelines. “It is often easiest to deal with clinical management of disease, especially pharmaceutical interventions. This is an area of health where you have the highest quality of evidence because you can do randomized controlled trials, which score highly in GRADE,” Penn says.In addition, qualitative research – or research on issues for which it is not possible to conduct randomized controlled trials – is often considered to be of low quality, which some researchers can take issue with, says Penn. But this is an evolving process, he says. “WHO is constantly looking at how to strengthen the GRADE methods as well as ways in which other types of evidence can be assessed and the applicability of GRADE can be broadened.”Once the guidelines have been developed at the global or regional level, WHO then needs to assist countries in adapting guidelines to their context. The consolidated HIV treatment guidelines are a case in point.For Dr Peter Cherutich, director of HIV prevention at the National AIDS & Sexually Transmitted Infections Control Programme at Kenya’s Ministry of Health, whose job it is to implement HIV guidelines, the new set of harmonized HIV treatment guidelines were useful – not least because they took into account the situation on the ground in the countries that were most likely to use them.“WHO has done an excellent job in guiding countries. The challenge has been the complexity of local considerations, in terms of cost, training needs and the diversity of local stakeholders – since the engagement of civil society in the implementation of clinical guidelines is still a new concept in Kenya – and the sheer time it takes to cascade the guidelines to lower level facilities,” Cherutich says. “WHO has a clear framework for doing this and has been able to mobilize other partners to assist Kenya to roll them out.”WHO guidelines can often be a catalyst in driving a policy change that a country is already contemplating, says Easterbrook, who co-led the development of the consolidated HIV guidelines.At the same time, when WHO issues new guidance, some countries may only just be implementing policy from the previous set of guidance.This can prove problematic at the treatment level, says Dr Halima Dawood, head of infectious diseases at Greys Hospital in KwaZulu-Natal, South Africa, which provides tertiary care and clinical support to HIV treatment clinics in western KwaZulu-Natal. “In South Africa, WHO guidance can be difficult to implement if it has not been sanctioned by the local department of health.”“WHO guidance can be difficult to implement if it has not been sanctioned by the local department of health.”Halima Dawood“WHO needs to better monitor the implementation of guidelines, to find out what works and what doesn’t, before proposing new guidelines,” agrees Pillay. “While science must lead, the issue of capacity to implement new guidelines is also important.”Input on the feasibility of guidance has long been incorporated during the guidelines development process. But the consolidated HIV treatment guidelines were different, as they draw on input from 120 stakeholders – including clinicians, researchers, implementers and community representatives – and more than 100 peer reviewers. “We collated thousands of comments on the guidelines,” says Easterbrook.When the consolidated HIV guidelines document was being disseminated last year, WHO held regional workshops at which Easterbrook and her colleagues from the HIV department at WHO headquarters in Geneva mapped treatment policies that countries had implemented or were planning to implement.AIDS in March showed that, by November 2013, 68 countries were planning to bring their policies into line with the new treatment recommendation to change the threshold, so that they would initiate ARV treatment for patients with a CD4+ count of 500 cells/microlitre or less, irrespective of clinical status – a major change from the previous guidelines that recommended starting treatment for a CD4+ count of 350 cells/microlitre or less.An evaluation by the HIV department and others published in a supplement of the journal Increasingly seeking public input into the development of guidelines, WHO has started opening up its guidelines to public consultation. For example, in March WHO invited comments on revised sugar guidelines.“The breadth and depth of the scope of WHO guidelines means that one size does not fit all, and some of the well-established methods may not be easily applicable to some subjects,” Penn says, adding that, ultimately, “the aim is to adhere to the core principles of best use of evidence, objectivity, freedom from bias, transparency and usefulness to the end user.”


Evidence-based medicine is the integration of individual clinical expertise, best external evidence and patient values which was introduced more than two decades ago. Yet, primary care physicians in Malaysia face unique barriers in accessing scientific literature and applying it to their clinical practice.This study aimed to explore the views and experiences of rural doctors’ about evidence-based medicine in their daily clinical practice in a rural primary care setting.Qualitative methodology was used. The interviews were conducted in June 2013 in two rural health clinics in Malaysia. The participants were recruited using purposive sampling. Four focus group discussions with 15 medical officers and three individual in-depth interviews with family medicine specialists were carried out. All interviews were conducted using a topic guide and were audio-recorded, transcribed verbatim, checked and analyzed using a thematic approach.Key themes identified were: (1) doctors viewed evidence-based medicine mainly as statistics, research and guidelines, (2) reactions to evidence-based medicine were largely negative, (3) doctors relied on specialists, peers, guidelines and non-evidence based internet sources for information, (4) information sources were accessed using novel methods such as mobile applications and (5) there are several barriers to evidence-based practice, including doctor-, evidence-based medicine-, patient- and system-related factors. These included inadequacies in knowledge, attitude, management support, time and access to evidence-based information sources. Participants recommended the use of online services to support evidence-based practice in the rural settings.The level of evidence-based practice is low in the rural setting due to poor awareness, knowledge, attitude and resources. Doctors use non-evidence based sources and access them through new methods such as messaging applications. Further research is recommended to develop and evaluate interventions to overcome the identified barriers. Evidence-based medicine (EBM) is the “integration of best research evidence with clinical expertise and patient values” . It is tIn rural areas, the concentration of poverty, low health status and high burden of disease create a need for greater attention to healthcare provision . Access However, there are challenges in adopting EBM in rural practice. These include greater time pressure , inappliThis study is part of a larger project, the Frontline Equitable Evidence-based Decision-Making study (FrEEDoM), which aims to develop an intervention to assist doctors in retrieving evidence to assist clinical decision making. The first phase of this project is to determine the views and experiences of the doctors in a rural healthcare setting regarding EBM. We aimed to explore their attitudes, barriers and needs with regards to the incorporation of clinical evidence into daily practice.A qualitative methodology was chosen to allow the exploration of participants’ personal views and experiences. Semi structured in-depth interviews (IDIs) and focus group discussions (FGDs) were used. Participants were family medicine specialists (FMS) who are doctors with postgraduate qualifications and MO who are doctors without postgraduate qualifications. We interviewed the FMSs individually because medical officers who worked under them may have felt inhibited in the discussion if they were in the same focus group.The interviews were conducted in June 2013 in two rural districts (Maran and Bentong) in Pahang—the largest state in Peninsular Malaysia. Rural was defined as areas with a population of less than 10,000 people with agriculture and natural resources  or areasWe obtained ethical approval for the study from the National Medical Research Register—Medical Research Ethics Committee (MREC reference: NMRR ID NMRR-12-1262-14539 S2 R0). All participants received RM100 (USD 30) as reimbursement for their time and travel expenses.Maran and Bentong District Health Offices provided contact details of the doctors serving at the district clinics. All 46 doctors working in public healthcare clinics in the two districts were invited via emails and phone calls to participate in the study. Of these, 18 participants from seven clinics  were recruited. Four FGDs with 15 medical officers and three individual in-depth interviews with FMSs were carried out. The interviews were conducted in two rural health clinics which was located at Karak and Maran Health Clinics by experienced facilitators (NCJ and LSM). Both are academic primary care physicians and advocates of EBM. All participants received and went through the participant information sheet before giving written consent. They were also assured of anonymity and confidentiality of the data. The demographic and clinical backgrounds of the participants are presented in The interview questions were developed based on the theory of planned behaviour (TPB), which is an extensively used psychological model for understanding human behavior . It infeAll the interviews were audio-recorded, transcribed verbatim and checked. The qualitative research software, NVivo10 was used to help sort, arrange and classify data. These data were analyzed using thematic analysis. Coding was done by a researcher with a nursing degree (NAH), a researcher with a degree in biomedical sciences (HGJ) and a health sciences librarian (RH). The data were read several times for familiarization and immersion. Text was labeled line by line to form the nodes. Nodes with similar themes were grouped to form categories. The coding was checked independently by LSM and NCJ, and discrepancies were resolved via discussion and consensus.The rest of the transcript was coded using the framework. Field notes were used to triangulate the data. Themes were then refined by removing or recoding redundant and infrequent nodes. These were then rechecked by LSM and NCJ to produce the final themes.We identified three themes in the context of practicing EBM in the rural setting: (1) doctors’ views and experiences in practicing EBM, (2) their barriers to incorporating EBM in the daily practice, and (3) their needs in order to facilitate evidence-based practice.Doctors associated EBM with statistics, clinical practice guidelines (CPGs) and evidence from research. It was believed that research and clinical trials produced the best evidence that can then be trusted and followed.“Evidence means that when you want to treat a patient, you have to make sure that the management that you give actually depends on research data. But CPGs are also research-based. Somehow it’s just more reliable, available or acceptable in our community to be used.” The participants expressed mixed attitudes toward EBM. Some felt that EBM was helpful in clinical decision making. Doctors felt that the EBM is important because good research evidence was seen to be universal; they can access the same source and share knowledge with doctors elsewhere.“I treat it [EBM] as the definite goal. This is what we really want for our patient, what we can offer as a substitute.” “My opinion is, I think it’s [EBM] really good, I mean, if there is an evidence-based resources then I will be more confident using it because it makes a lot of sense. Because it’s based on the evidence so I find it rather useful.” However, not all participants had the same opinion; some felt that EBM was not important or necessary to clinical practice. Many felt frustrated because they were not taught about EBM in depth during medical training. They were negative about the practice of EBM in their daily work. EBM was perceived as requiring the use of a lot of resources such as time, reading and skills.“Actually the first thing that I know about EBM is you have to read a lot. You have to spend more time and be more passionate to read journals in order to be knowledgeable, well…it’s just that we are not really a reading nation…”“Because we don't understand what is it [EBM] and then when we have the interest, of course we will go through; if we have no interest, we try to skip.” One doctor stated that EBM came second, after clinical experience. With a background in public health, he perceived that research outcomes and statistics are easily manipulated.“Certain things you have to follow the evidence…let’s say if you don’t have knowledge about the management, then it’s better to follow evidence-based. But sometimes certain things you already know the practice better, then it’s fine. I think the evidence comes second.”During the interviews, participants were probed on the practice of EBM in their daily clinical routine. The participants spoke about their actual use of a mobile application which allows them to broadcast their clinical question in real time to a group of their peers. The mobile application, WhatsApp, was used by the participants because it allowed immediate response from their peers. Their peer group would comprise colleagues in different specialties; those considered ‘experts’ such as a medical officer working in a hospital will be able to give answers based on their current practice. This was a form of ‘peer-to-peer’ information sharing that took place over an electronic online resource. This information exchange, though not secure, is usually limited to a selected group of participants and is not accessible to the public.“We have a good discussion group, so we use our WhatsApp usually for our discussion such as picture sharing. It’s easy to discuss, for example we can just snap a photo of the X-ray of some fractures that we are not sure of the management and send it via WhatsApp. Sometimes if you ask a silly question also they will answer because we are friends.” “WhatsApp is the best thing ever designed I guess! So usually when we call them to refer anything, the MOs will ask us to send the picture, you know last time we used to use MMS (Multimedia Messaging Service), but I guess it took some time and then sometimes we have a problem with it but WhatsApp is one of the easiest thing.” “We discuss with our specialist. As I told you they’re not the main concern, but the main priority is that the pediatrics and antenatal. This two is like a national mortality. So when it comes to these two, we just called up the specialist and then just discuss with them.” Participants still relied on CPGs  as a source of information on managing their patients. Many felt that these recommendations published by the Ministry of Heath, Malaysia were based on well-appraised evidence and should be adhered to. Doctors also mentioned international guidelines as a source of information, e.g., NICE guidelines.“Other people have made it [CPG] easy for us to implement evidence-based medicine.” “We can depend on solid facts based on this CPG and we can always use it. We feel safe to follow this and if something happened to the patient based on the management that we’ve done, that means you are not in trouble you see? Rather than, oh like I say we have the same situation and Dr F will go with the EBM, I’ll go with the CPG, I think I will I can sleep better at night. I think I can sleep better rather than Dr F, it’s I think how it works here. How people take things. Some of the doctors shared their experiences in using EBM in daily clinical practice. Apparently, most of the doctors were relying on pre-appraised evidence, which were mainly the guidelines to seek quick answers on patient management.“Usually we use Clinical Practice Guidelines or the evidence-based guidelines as our guide. I will take pictures or jot down the management and if I encounter this kind of cases, then I will know what to do first. You should look for this [management] then after that you think whether the management of the disease is possible. This is how I practice, but usually you need time to go and read up.” Participants also consulted friends and colleagues to get immediate response and solutions to clinical problems.“Because if we are not sure then we will discuss among ourselves and then if still can’t get answers, then we will ask our boss and the specialists.” “We have friends and colleagues’ who work in hospitals, we share what we do here in primary care and how is their management in tertiary settings. They also have their specialist to consult…” Participants generally considered the internet as one of the more important sources of information. The websites that they frequently used were Wikipedia, Medscape, WebMD and e-Medicine. Only a handful of doctors mentioned medical databases such as PubMed and UpToDate. “Googling” is the preferred means to search for information.“Usually I would search for the answers through internet or maybe I find the information from Google.com. I will search whatever I want to find. I will choose reliable websites such as e-medicine or some journal.” “It depends on what type of clinical questions, usually I will just refer to the CPG, if there’s no CPG, I will usually go to Medscape” The barriers that emerged can be categorized into doctor, EBM, patient and system barriers.Participants felt that they faced a lot of difficulties in practicing EBM. The main problem was lack of time, limited exposure to EBM and skills. Doctors also struggled to implement EBM in their daily practice because it required considerable effort. Training was limited in medical schools. Most of the training provided focused on statistical knowledge rather than teaching students on how to apply evidence to patient care in the ‘real world.’“We only had little exposure to it [EBM]. Because we just studied for the examination purpose and they just gave us a project and you are required to sum it up, do all the confidence interval and all those kind of thing, just one project about it.” This barrier was mainly due to the perceived image of EBM itself. The term of EBM was felt to be intimidating. One participant perceived EBM to be a measure used by health authorities to check on his work quality.… evidence-based …maybe they want to ask me questions what’s happening… like an audit. “EBM sounds like an assessment of our knowledge… whether you are good doctors or not. Very scared…” Evidence was also seen to be unreliable because it changed over time.“EBM was also too volatile; mobile too. I don't know I can use it …dynamic you know? Ever changing …regularly comes up with new [evidence]…” EBM was also seen as being irrelevant to rural settings. Participants also made the observation that researches were mainly conducted in urban settings and felt that results cannot be translated to rural practice.“To me sometimes evidence-based medicine cannot [be] applied to our local situation, maybe most of the research done at urban area settings so they have their own guidelines, less research conducted in Pahang.” The participants found it difficult to practice EBM when their patients preferred to seek alternative treatment, self-medicate or self-prescribe for their illnesses. The doctors felt helpless as these factors are perceived to be out of their control.“In FELDA [Malaysian rural land development scheme] areas, most of the patients are peneroka [settlers], so most of them take the traditional medicine. Problem konon [as though] patient cannot agree with our medicine “Like those orang asli [aborigines] they have their own belief, their rituals and everything, they say ‘Oh I don't want to go and see any doctor, I already did jampi [spell], I already went to see bomoh [shaman] …my kid inside is protected, they don't want to do their antenatal care’…”  …”Patients tended to ignore medical advice, even though it might be evidence-based. Participants felt that the community was not aware of EBM and refused recommended practice, making it harder for doctors to also practice EBM.“Another main issue is patients self-medication. They can walk in and buy anything they want from the pharmacies, the higher end antibiotic to lower end antibiotic, anything there is. Can go walk into a pharmacy and then patient only come to us when they already have high fever. They already have taken their own medication- antibiotic, they go to the GP clinic, then after 2 days jump to another GP! Without …they don't have any blood taking. I’m doing locum also. GP like me also sometimes I never give medicine that the patient wants, they will jump to another clinic. So, to avoid that to happen, I just give you my medication. After two to three times go to the GPs, then they will come to KK [Klinik Kesihatan ], by that time conditions already worsen.” “I think rather than just educating the doctors, we should educate the community too. Like for example, drugs are available in the shop [pharmacies] and you advise people don't take but we are selling it. System! Yes, there is a problem with the system! “There were several factors arising from the healthcare system and policies that impede the practice of EBM in the rural primary care setting, and they include heavy workload, lack of resources, difficulty in accessing information and variations in clinical standards and practice.Time constraint and heavy patient load were cited as major hindrances in practicing EBM.“It’s not like one person we are taking care of, the ratio here is crazy for Mempaga and Karak and sometimes it’s really crazy! 1 to 100 and all the cases are complicated you know.” “It’s difficult sometimes to practice based on all this research, evidence and all, when we have to search then it is going to be quite difficult…[Laughed] because we have lots of patients lining up outside. It’s going to take some time and just because people like to complain.” The health clinics lacked facilities and resources that were seen as essential for the doctors to practice EBM. For instance, there was a limited choice of medications available in the rural health clinics.“We are in KK [health care clinics], especially KK we don't have much medicine. Then, we admit patients. Our resources are very limited. So we cannot admit the patient and we cannot send the patient back home also because when we send the patient back home, we don't know what's going to happen at home. Whether they are going to follow what we say or whether they take their medication or not, we don't know. So, the best is, we call up the specialist, talk to them and send them to the hospital.” “But usually we won’t be able [to practice EBM] because we have so limited resources back in clinics. Mmm because based on evidence of course the best medication would be this, this or this and yet we have limited drugs and we only have this; by hook or by crook we have to manage our patients with the resources that you have.” The doctors also had difficulty in accessing the information in their rural practice. The internet connection was unreliable and the doctors mostly relied on their personal internet service provider using their mobile phones. Accessing the full-text publication was also difficult because the organization did not subscribe to these databases.“Connection is quite a problem.” “Soft copy yes, but hard copy I rarely use it now and then it [journal] is expensive to get and sometimes they just give the summary. We cannot download and if need to buy it, I don’t subscribe to it.” Most of the participants wanted to have a more reliable internet connection to search for information and practice EBM. Without that, it would seem useless for them to consider interventions to improve the practice of EBM.“I think with better internet access and proper specific websites for us, you know some websites that we can go look specifically for this kind of thing.” One of the FMSs felt that teleconferencing would be helpful. She had experienced journal club by teleconferencing as a medical student during her rural posting abroad. It allowed the sharing of information and continuing medical education with colleagues in remote settings.“It would be nice if you have the teleconference. Everybody will actually receive it. Ok, so that might work better. The Skype will solve the transportation problems.I find it quite useful.” Some doctors wanted to share information publicly through forums or via Facebook.“Yes! I think if you were to try via Facebook, it would be the best….we will get notification for the answers given right.” “Because for example, nowadays everybody has Facebook account, even our Prime Minister has a Facebook account to communicate with people right? So for us to practice EBM sometimes we get the experience by making friends with doctors, senior doctors and consultants through Facebook, sometimes they give quite useful information.” Some requested an all day telephone helpline service to support doctors in decision making. Participants wanted immediate answers, especially with regards to emergency cases.Doctors were asked for suggestions on the development of an EBM support service. They recommended the use of online delivery modes including websites, email service, and forums to support them in practicing EBM in their rural practice.“Webpage with frequent answers would be useful. If it’s in an email, we’ll read. If it’s not in the email then we wouldn’t know what’s the update. We wouldn’t know when it’s going to be updated. Well, in a way that we can answer the questions and then you reply us with the answer and then you attached the published paper as an extra information.”“Forum. Everyone can give their own opinion, so those who have created the website can give positive feedback, so other doctors they can join in to give the opinion …” The key findings from this study show the complexity and challenges of practicing EBM in a rural setting. In the discussion, we will first fit the findings to the theoretical framework. Then we will discuss the five main themes found from the study namely that (1) doctors viewed evidence-based medicine mainly as statistics, research and guidelines, (2) reactions to evidence-based medicine were largely negative, (3) doctors relied on specialists, peers, guidelines and non-evidence based internet sources for information, (4) information sources were accessed using novel methods such as mobile applications and (5) there are several barriers to evidence-based practice, including doctor-, evidence-based medicine-, patient- and system-related factors.We applied the findings to the theoretical framework as shown in The first theme we found was that doctors viewed evidence-based medicine mainly as statistics, research and guidelines. The definition of EBM is based on the three pillars; individual clinical expertise, patient values and expectation, and the best external evidence. Participants defined EBM mainly as research evidence and they also considered clinical practice in the definition. However, none of the interviewed doctors mentioned ‘patient values’ as a component of EBM. Patient values are defined as “the unique preferences, concerns and expectations each patient brings to a clinical encounter and which must be integrated into clinical decisions if they are to serve the patient” . This glSecond, doctors expressed mainly negative attitudes toward the implementation of EBM. Awareness of EBM was prevalent but it was seen as unimportant and irrelevant. Participants in this study perceived EBM as creating more workload and their work setting as lacking in resources. The most recent study conducted in the United Kingdom found that doctors were generally positive toward the practice of EBM and their knowledge about EBM was higher compared to the previous published study ,25. In aThe third and fourth theme are related in that participants still preferred obtaining information from their peers and interestingly, the study participants used WhatsApp—a smart phone messenger—as a means of doing so. Searching information other than from peers was limited to accessing informal websites by Google search such as Wikipedia. The rapid rise in the use of smart phones has enabled advanced mobile communication between healthcare professionals. Smart phones provide access to evidence-based medical resources including disease diagnosis guides, drug references, literature search, and continuing medical education materials at the point of care . Real-tiFinally, the barriers identified in our study were similar to those published previously such as insufficient time to access the sources, insufficient basic EBM skills, lack of time, lack of resources, lack of EBM training workshops and courses . HoweverAlthough there had been a number of studies conducted in which many of the barriers toward EBM were similar, this study highlights that progress was still limited in the rural setting in Malaysia. Attitudes and knowledge have remained poor despite the fact that EBM had been introduced for more than two decades. This exploratory method using a qualitative design allowed factors unique to the setting to be identified and understood. Until unique barriers were made known, interventions to tackle the challenges would not be successful. The barriers and needs identified in this study would have particular significance and value to settings with limited resources.This research highlighted the views and experiences of the doctors working in rural areas where the setting imposed different challenges from the better studied urban settings. The qualitative design allowed a multi-dimensional perspective of the data obtained from its natural setting. We believe that this explorative approach will better inform the development of interventions to overcome the barriers to the practice of EBM.This study had some limitations. There are areas in Malaysia that are even more remote and rural where there may be greater barriers and challenges. This study was originally planned to be conducted in rural areas in Sabah in East Malaysia where health inequity issues are a problem. Due to the distance and greater resource requirements, it was decided to conduct the interviews in the state of Pahang, Peninsular Malaysia. However, it is unlikely that findings would be significantly different in the state of Sabah, as the doctors in both states work under the same healthcare system.In this study, the researchers’ personal views and experiences might have had an influence on data analysis. NCJ and LSM are academic primary care physicians and advocates of evidence-based medicine. To minimize this bias, data were transcribed verbatim, coded and analyzed by RH, NAH and HGJ. The results were then checked and counterchecked with all members.The barriers affecting the EBM implementation identified in this study should be targeted for intervention. This can be researched through interventional trials. We also recommend that this study be replicated in regional urban settings to investigate whether the doctors’ views and barriers are different. In clinical practice, skills for EBM can be enhanced through carefully targeted, multidisciplinary undergraduate and continuing education programs, especially for rural doctors. Evidence-based guidelines need to be locally adapted to attend to the context of rural practice. Doctors should be encouraged to involve patient values in their practice. The implementation of EBM is likely to be more successful if it is practised in ways that doctors are using currently.Most primary care physicians in rural health clinics in Malaysia do not use EBM. Factors associated with the non-use of evidence can be categorized into doctors’ barriers, EBM barriers, patient barriers and system barriers. This included lack of knowledge about EBM, attitudes and intentions toward the implementation, lack of support and encouragement from the management, lack of time to practice EBM, and insufficient access to relevant information sources. Further research is recommended to develop and evaluate interventions to overcome these identified barriers.S1 Appendix(DOCX)Click here for additional data file.


Central obesity has been suggested as a risk factor for gastroesophageal reflux disease. The aim of this study was to evaluate the association of visceral fat area and other lifestyle factors with reflux esophagitis or Barrett’s esophagus in Japanese population.Individuals who received thorough medical examinations including the measurement of visceral fat area by abdominal computed tomography were enrolled. Factors associated with the presence of reflux esophagitis, the severity of reflux esophagitis, or the presence of Barrett’s esophagus were determined using multivariable logistic regression models.2; 95% confident interval, 1.01 to 1.46) and women . Current smoking and serum levels of triglyceride were also associated with the presence of reflux esophagitis in men. However, significant association between visceral fat area and the severity of reflux esophagitis or the presence of Barrett’s esophagus was not shown. In men, excessive alcohol consumption on a drinking day, but not the frequency of alcohol drinking, was associated with both the severity of reflux esophagitis  and the presence of Barrett’s esophagus .A total of 2608 individuals were eligible for the analyses. Visceral fat area was associated with the presence of reflux esophagitis both in men (odds ratio, 1.21 per 50 cmVisceral fat area was independently associated with the presence of reflux esophagitis, but not with the presence of Barrett’s esophagus. On the other hand, quantity of alcohol consumption could play a role in the development of severe reflux esophagitis and Barrett’s esophagus in Japanese population. Esophageal adenocarcinoma is an aggressive malignancy with a poor prognosis. In the Western countries, the incidence of esophageal adenocarcinoma has increased dramatically in the last three decades. AlthoughBarrett’s esophagus (BE) is the presumed precursor lesion of esophageal adenocarcinoma. Previously diagnosed reflux esophagitis (RE) also increases the risk of esophageal adenocarcinoma. In the WSince the present study was non-intrusive retrospective research, informed consent for each participant was not obtained. The records of participants were anonymized and de-identified prior to analysis. The protocol for this study was approved by the ethics committee of the Keio University School of Medicine (No. 20130501).Individuals who received esophagogastroduodenoscopy (EGD) and thorough medical examinations at Keio University Hospital between October 2012 and November 2013 were enrolled. Thorough medical examinations included questionnaires of medical history and lifestyle habit, physical examination, complete blood count, serum chemistries, computed tomography (CT) scans of the chest and the abdomen, abdominal ultrasound, and EGD. Individuals who had undergone esophageal or gastric resection before the medical examinations were excluded from the study. Those with incomplete questionnaires were also excluded.RE was diagnosed and graded according to the Los Angeles classification. BE was d2), waist circumference at the level of the navel during minimal respiration, cigarette smoking status, alcohol consumption, medication, and sleeping hours from the questionnaires. In terms of alcohol consumption, frequency of alcohol drinking (number of days per week) and quantity consumed on days during which drinking occurred (gram ethanol per drinking day) were asked separately. The frequency and intensity of daily exercise, such as sweating for more than 30 minutes twice a week or walking for one hour or more in a day, and eating habits, such as skipping breakfast 3 times a week, having late evening snack 3 times a week, and having late dinner 3 times a week, were also asked using questionnaires. Blood pressure and blood markers including triglyceride (TG), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), hemoglobin A1c (Hb-A1c), homeostatic model assessment of insulin resistance (HOMA-R), and uric acid (UA) were measured. In addition, visceral fat area (VFA) was measured at the navel level using a single-slice CT scan .We obtained information on age, sex, body mass index  in the same way.Analyses were conducted for each gender separately. Factors associated with the presence of RE, the severity of RE (Los Angeles grade A vs. B to D), or the presence of BE were evaluated. For univariable analyses, continuous variables were compared using Student's t-test. The chi-square test or Fisher's exact test was used to compare categorical variables. Subsequently, factors that showed marginal differences (p < 0.2) in univariable analyses were included in age-adjusted logistic regression models. The comparison unit for a continuous variable was set to be close to the standard deviation of each factor. Finally, marginally associated factors (p < 0.1) in age-adjusted logistic regression analyses were included in multivariable logistic regression models with forward stepwise variable selection. Subgroup analyses were also conducted based on VFA .A total of 2,783 individuals underwent thorough medical examinations including EGD during the study period . After wDifferences between individuals with and without RE were investigated using the univariable analyses . IndividThe rate of current smokers was higher in those with RE than in those without RE in men, but not in women. The rate of individuals who consumed more than 40 g ethanol on a day in which alcohol was consumed tended to be higher in those with RE than those without RE in men (p = 0.06), but not in women (p = 0.63). However, frequency of alcohol drinking was not associated with the presence of RE.The rate of individuals who had a late dinner 3 times a week or more was higher in those with RE than without RE in men. The rate of individuals who slept less than 6 hours a night was also higher in those with RE in men. The presence of hiatus hernia or SSBE was associated with the presence of RE both in men and women.Differences of characteristics between individuals with grade A esophagitis and those with grade B–D esophagitis were investigated using the univariable analyses . In men,Differences of characteristics between individuals with SSBE and those without SSBE were investigated using the univariable analyses . In men,2; 95% confident interval [CI], 1.00 to 1.46), levels of TG , current smoking , and the presence of hiatus hernia  were independently associated with the presence of RE. More than 40g ethanol of alcohol consumption on a drinking day , the presence of hiatus hernia , and the presence of BE  were independently associated with the severity of RE. More than 40g ethanol of alcohol consumption on a drinking day  and the presence of hiatus hernia  were also independently associated with the presence of BE. In women, levels of VFA , the presence of hiatus hernia , and the presence of BE  were independently associated with the presence of RE. In terms of the severity of RE, no factors could be extracted in women. Sleeping less than 6 hours a night , the presence of hiatus hernia , and the presence of RE  were independently associated with the presence of BE.Using age-adjusted logistic regression analyses and subsequent multivariable logistic regression analyses, we could identify independently associated factors for the presence or the severity of RE or the presence of BE in men  and wome2, independently associated factors for the presence or the severity of reflux esophagitis or the presence of Barrett’s esophagus were also evaluated  and the presence of SSBE  was associated with the severity of RE. The presence of hiatus hernia  was associated with the presence of BE. In women without visceral fat accumulation, levels of VFA , the presence of hiatus hernia , and the presence of SSBE  were independently associated with the presence of RE. The presence of hiatus hernia  and the presence of RE  were independently associated with the presence of BE. In women with visceral fat accumulation, the presence of hiatus hernia  was associated with the presence of RE. Factors associated with the severity of RE in women with or without visceral fat accumulation, and factors associated with the presence of BE in women with visceral fat accumulation could not be identified.After the participants were subcategorized into the presence or absence of abdominal fat accumulation using the cut-off point of VFA at 100 cmvaluated . The res2). This result suggests that abdominal visceral adipose tissue not only mechanically disrupts the integrity of the gastroesophageal junction barrier, but also has some endocrinological effects on the esophagus. According to previous reports, serum levels of inflammatory cytokines and adipokines, such as adiponectin, leptin, IL-6, and IL-1β, were associated with the presence of RE.[The present study showed that visceral fat accumulation was associated with the presence of RE in both men and women. Especially in women, increasing levels of VFA was an independent risk factor among participants without visceral fat accumulation  (Table G). Factors associated with reflux esophagitis or Barrett’s esophagus in obese men  (Table H). Factors associated with reflux esophagitis or Barrett’s esophagus in non-obese women  (Table I). Factors associated with reflux esophagitis or Barrett’s esophagus in obese women  (Table J).Differences between participants with and without reflux esophagitis in non-obese and obese men (Table A). Differences between participants with and without reflux esophagitis in non-obese and obese women (Table B). Differences between participants with mild and severe reflux esophagitis in non-obese and obese men (Table C). Differences between participants with mild and severe reflux esophagitis in non-obese and obese women (Table D). Differences between participants with and without short-segment Barrett’s esophagus in non-obese and obese men (Table E). Differences between participants with and without short-segment Barrett’s esophagus in non-obese and obese women (Table F). Factors associated with reflux esophagitis or Barrett’s esophagus in non-obese men Click here for additional data file.S1 Raw Data(SAV)Click here for additional data file.


Perceptual decisions are thought to be mediated by a mechanism of sequential sampling and integration of noisy evidence whose temporal weighting profile affects the decision quality. To examine temporal weighting, participants were presented with two brightness-fluctuating disks for 1, 2 or 3 seconds and were requested to choose the overall brighter disk at the end of each trial. By employing a signal-perturbation method, which deploys across trials a set of systematically controlled temporal dispersions of the same overall signal, we were able to quantify the participants’ temporal weighting profile. Results indicate that, for intervals of 1 or 2 sec, participants exhibit a primacy-bias. However, for longer stimuli (3-sec) the temporal weighting profile is non-monotonic, with concurrent primacy and recency, which is inconsistent with the predictions of previously suggested computational models of perceptual decision-making (drift-diffusion and Ornstein-Uhlenbeck processes). We propose a novel, dynamic variant of the leaky-competing accumulator model as a potential account for this finding, and we discuss potential neural mechanisms. An important process that supports decision-making is the integration of evidence over time, which optimizes decision quality by enhancing the signal to noise ratio. The nature of this process depends critically on the weight given to evidence across time: which information has more impact—early, intermediate or late? We used a novel psychophysical technique, which relies on differential temporal dispersion of evidence. This technique allowed us to extract the temporal weights people assign to the flow of evidence. We find that in decisions that are based on relatively short streams of evidence, people gave stronger weight to early information (primacy). Surprisingly, however, with longer streams of evidence, people assigned higher weights to early and late evidence, while underweighting intermediate evidence. This non-monotonic pattern of evidence integration is not predicted by the existing models of decision-making, posing a challenge to current theories. We propose a novel model that accounts for non-monotonic weighting, based on a change in leak and response-competition with integration-time, and we discuss potential neural mechanisms. Temporal integration of noisy evidence is a central component of the mechanism that mediates perceptual and value-based decisions –13. In pIn the present study we probe the temporal weighting of evidence over a wider range of durations . We do so by using perceptual evidence-samples that are longer 100 msec) than the ones used in the moving dots , 18 or o0 msec thThe central aim of this study is to probe the temporal weighting profile that participants assign to evidence-samples under these expanded conditions. To anticipate our results, we find a complex temporal-weight profile, which at longer durations is non-monotonic in time. As we show in the next section, such temporal profiles provide strong constraints on computational models of perceptual choice. A secondary aim is to probe the extent to which participants are able to carry out temporal integration in these conditions. In previous studies of perceptual decisions the temporal extent of this integration was relatively limited  in perceptual decisions. Next, we present the results of three experiments which reveal a non-monotonic temporal weighting profile at longer durations. We then present a novel computational model, which extends the Leaky Competing Accumulator (LCA) to include a dynamic variation of its parameters across time. Finally, we show that it can account for the temporal weighting profiles, discuss its underlying neural mechanism, and we highlight some alternative accounts.A number of computational models have been proposed to account for binary decisions based on fluctuating evidence . One of these models, the drift–diffusion model , the inAnother group of perceptual-choice models assumes competitive interactions between cell assemblies that correspond to the choice alternatives. Examples of such models include the leaky-competing accumulator model  and theThe LCA consists of two accumulators, one for each alternative, which compete with each other via lateral inhibition and are subject to decay (or leak) of activation. Here like in the standard drift-diffusion model, the evidence is integrated without a boundary in interrogation paradigms, and the decision is determined in favor of the accumulator whose activation is the highest at the end of the integration interval. Importantly, in the LCA, the ratio between lateral inhibition and leak determine the shape of the temporal weighting profile.As we illustrate in The DDM with an absorbing boundary predicts primacy  temporal weights can arise in two computational models that account for the mechanism by which observers integrate evidence and trigger decisions. The aim of the experimental study was to test how the temporal weight profile depends on the duration of the evidence. As we will show, the results provide a challenge to all the models described above.To probe for temporal weighting biases we have incorporated a method of signal-perturbation , 37, in Based on previous behavioral and computational studies described above we expected to observe either a monotonic decreasing temporal weighting profile (primacy), or, to a lesser extent, a monotonic increasing profile (recency).Participants’ performance (accuracy-Pc and mean-RT) did not differ between baseline and perturbed-signal conditions , as well as between congruent and incongruent trials, when averaged across all time-windows ; as we show below, the difference in accuracy between congruent and incongruent perturbations varies across time-windows.In order to determine the extent of temporal integration of evidence we first examine the dependency of the accuracy on the duration of the evidence . We observe that the accuracy improved with trial-duration over the full 1–3 sec interval , suggesting that participants integrate the perceptual evidence . Note, tWhile modest (~10%), this extent of integration, extending for 30 samples, corresponding to 3 seconds of noisy evidence, is predicted by the model we propose below. This exceeds by almost an order of magnitude, the temporal extent observed in the moving dot paradigm  were slower for longer trials (3-sec), as compared to the 1- and 2-sec trials , indicating that participants did not prepare their response in advance of the evidence termination. If they did so, one would expect faster RT at longer duration, since a larger fraction of the trials may have reached an implicit decision .i denotes the temporal window ; and ii) a logistic regression on observed choices with each of the temporal window’s average signal as predictors (defined as the difference between the two brightness levels). As the two measures give identical conclusions, we will focus here on the behavioral measure.We quantified temporal integration biases, using two measures: i) a behavioral index of the influence of the perturbation on choice probability as compared to baseline, as a function of its temporal window, given by: As shown in st temporal window was higher than in the 2nd window . The influence of the signal arriving at the 1While the fraction of participants whose temporal weight at window-2 is lower than at window 1 and 5, is significantly higher than expected by chance , the identity of these critical windows was selected based on the data, and thus cannot provide valid statistical test. For that reason, we conducted two additional experiments, in which we sought to replicate this unexpected pattern of temporal weighting by presenting solely 3-sec trials , as well as randomly varying trial-durations in order to additionally ensure that the observed non-monotonicity is not due to participants’ fatigue from repetitive trials of the same duration . On the basis of the previous results, we predict that the temporal weights for 3 sec evidence trials will be non-monotonic, with higher weights at window 1 and 5, compared with window-2.Experiment-2 (N = 10) was identical to Experiment-1, with the exception that only 3-sec trials were presented . Experiment-3 , was identical to Experiment-1, with the sole exception that trial-durations  were randomized rather than blocked.The temporal weights observed for the 1 and 2-sec trials in Experiment-3 were similar to those observed in experiment-1, both indicating a primacy bias . Thus, we have replicated the finding that under short presentation times perceptual decisions are primacy biased.The temporal weighting profile of the 3-sec trials did not differ between experiment-2 and experiment-3 , and is therefore reported below collapsed across both experiments  = 2.26; p = 0.036;], while the influence of the 5th window was also higher than the 2nd one . Note that the temporal resolution in this analysis is much higher than the one used in the behavioral perturbation design Click here for additional data file.S3 Figst vs. 2nd window; t(9) = 1.16; p = 0.27; 5th vs. 2nd window; t(9) = 1.91; p = 0.08; Exp. 3: 1st vs. 2nd window; t(9) = 1.99; p = 0.08; 5th vs. 2nd window; t(9) = 1.56; p = 0.15].In each experiment seperately, we find numerical trends of non-monotonic weighting functions [Exp. 2: 1(DOCX)Click here for additional data file.S4 FigParameters were manually chosen to meet the non-monotonic pattern observed in the 3-sec trials. As can be seen, a model that assumes that on a fraction of trials accumulation is recency-biased, and on other trials primacy-biased predicts non-monotonic weighting in 1- and 2-sec trails as well.(DOCX)Click here for additional data file.S5 FigAs can be see, observed weights are monotonically increasing indicating recency-biased integration. This pattern is predicted by the DLCA model. Simulation was conducted using the best-fitting parameters obtained for the 3-sec data.(DOCX)Click here for additional data file.S1 TableDescription of the parameter-space.(DOCX)Click here for additional data file.