Vectored vaccines based on highly attenuated modified vaccinia Ankara (MVA) are reported to be immunogenic, tolerant to pre-existing immunity, and able to accommodate and stably maintain very large transgenes. MVA is usually produced on primary chicken embryo fibroblasts, but production processes based on continuous cell lines emerge as increasingly robust and cost-effective alternatives. An isolate of a hitherto undescribed genotype was recovered by passage of a non-plaque-purified preparation of MVA in a continuous anatine suspension cell line (CR.pIX) in chemically defined medium. The novel isolate (MVA-CR19) replicated to higher infectious titers in the extracellular volume of suspension cultures and induced fewer syncytia in adherent cultures. We now extend previous studies with the investigation of the point mutations in structural genes of MVA-CR19 and describe an additional point mutation in a regulatory gene. We furthermore map and discuss an extensive rearrangement of the left telomer of MVA-CR19 that appears to have occurred by duplication of the right telomer. This event caused deletions and duplications of genes that may modulate immunologic properties of MVA-CR19 as a vaccine vector. Our characterizations also highlight the exceptional genetic stability of plaque-purified MVA: although the phenotype of MVA-CR19 appears to be advantageous for replication, we found that all genetic markers that differentiate wildtype and MVA-CR19 are stably maintained in passages of recombinant viruses based on either wildtype or MVA-CR.The online version of this article (10.1007/s12250-019-00176-3) contains supplementary material, which is available to authorized users. Modified vaccinia Ankara (MVA) is a highly attenuated poxvirus investigated as vector for protective and therapeutic treatment also of immunocompromised human patients  and developed chemically-defined media (Jordan 8 infectious units are estimated to be required for vaccination because MVA is not amplified (replicated) at the injection site   or baffled shake flasks , were equipped with 0.2 µm filtered lids to allow gas exchange. Suspension culture volumes were maintained at 20%–40% of the vessel size.CR.pIX cells were maintained in adherent format in DMEM:F12 medium supplemented with 5% bovine serum , or in suspension cultures in CD-U4 medium  supplemented with 10 ng/mL LONG-R3IGF . Both media were also supplemented with 2 mmol/L GlutaMAX I . Infection and propagation of MVA was performed in 1:1 mixtures of CD-U4 and CD-VP4 (Merck-Millipore #F9127) as described previously, usually with 2 × 10et al. Infectious titers of MVA were determined in PFU/mL (plaque forming units) or FFU/mL (fluorescence forming units) as described previously  according to the manufacturer’s instructions 90 min after infection, and the medium was replaced 24 h post transfection. The infected/transfected culture was harvested after 48 h to 72 h, sonicated, and used to infect cell monolayers in a 6-well plate. Plaques of the appropriate fluorescent phenotype were picked usually after another 24 h to 48 h and total DNA was isolated from aliquots of individual plaques using QuickExtract DNA Extraction Solution 1.0 . Another round of plaque purification was initiated with the candidate recombinant virus preparations that passed the PCR analysis. The material for infection was obtained by sonication of cell harvests using a Vial Tweeter (set to 20 s of 100% cycle and 90% amplitude) that allows handling of closed sample caps to avoid cross-contamination . Viruses with parental genotype or incomplete recombination were not detectable within 3–8 rounds of plaque purification.6 cells/mL and MOI 0.01 (in blind passages a titer of 108 PFU/mL was assumed in the previous passage). The infected culture was sonicated 48 or 72 h post infection to harvest virus.Virus passages to assay genomic stability was performed in CR.pIX suspension cultures in a volume of 5 mL with 1:1 mixtures of CD-U4 and CD-VP4. Cell density was 2 × 10et al. et al. 80 µL of complete cell lysate was mixed with 20 µL of QuickExtract DNA Extraction Solution 1.0  and heated to 65 °C for 10 min and to 98 °C for 5 min. 4 µL of this preparation was subjected to PCR in a final volume of 25 µL with 0.15 µL Taq polymerase , 200 nmol/L each primer, and 125 µmol/L each nucleotide. The sequence of the primer pairs that span deletion sites I–VI of the viral genome were obtained from the literature  and Pci I, the right flank with Dra III and Not I for sequential insertion into the same sites of pEGFP-N1 while maintaining the EGFP open reading frame. The artificial EL promoter . The flanks were amplified from the genomic DNA of wildtype MVA with the primers shown in Supplementary Table S1. The left flank was cut with Nhe I and Xba I, respectively, at the 5′ termini for insertion into pEGFP-N1 [out of dam(−) bacteria] using these sites. The region of interest for the L3L shuttle plasmid was also obtained by PCR and inserted into pCR-Blunt II-Topo (“pTopo”) as described in the Zero Blunt TOPO PCR Cloning Kit .The shuttle plasmids for introduction of the point mutations D86Y in A34R and V110A in L3L into wildtype MVA were cloned only with fragments amplified out of MVA-CR19 genomic DNA. These mutations already contain by chance diagnostic restriction enzyme sites to confirm successful recombination (Table http://resitefinder.appspot.com/).The shuttle plasmids for the other point mutations, H639Y in A3L and K75E in A9L, were cloned by insertion of synthetic DNA (Eurofins Genomic) that contained the desired point mutation and silent diagnostic mutations  Ava I site was appended to the Nco I-distal side using Swa I (in MVA) and Spe I (in pTopo).For the generation of the shuttle plasmid for A3L a 2008 bp fragment of A3L was amplified (primer sequences shown in Supplementary Table S1) and inserted into pTopo. A 274-bp fragment therein from Pme I (in the vector) to Nsi I (in the MVA insert) of pTA10L as new vector backbone, Spe I to Nsi I in the synthetic DNA to insert the silent Sty I diagnostic marker, and Nsi I to Pme I of the pTA10L to restore the initial amplification product. Three-fragment ligation to obtain pTLA10L-StyI was necessary to circumvent an additional Spe I site in the vector backbone. The A9L flank was inserted using a synthetic DNA fragment containing the K75E mutation framed by diagnostic silent mutations on both sides, EcoR I and BseR I. This fragment was inserted via a three-fragment ligation to circumvent a Tth111 I site in the vector, using Tth111 I in A9L to EcoR V in the multiple cloning site of the synthetic DNA vector, Tth111 I to Pme I in pTA10L and Tth111 I to Pme I in pTA10L to restore the vector. The resulting shuttle plasmid contains a MVA-derived fragment of 3620 bp.Recombination of a 415 bp synthetic DNA that also contained a silent diagnostic mutation near to the desired K75E mutation transferred only the diagnostic mutation as well . The flanks were therefore extended and additional diagnostic mutations were inserted so that K75E is framed by markers as follows: First, a 2905 bp fragment containing A9L and neighboring gene A10L was amplified out of wildtype MVA genomic DNA with primers shown in Supplementary Table S1. This 2905 bp fragment was cloned into pTopo to yield pTA10L. A synthetic DNA containing the diagnostic sites in A10L was then inserted via a three fragment ligation using 6 cells/mL as described previously  5′-end RACE was performed. Primer D2 RII (Supplementary Table S1) against a 5′ terminal region still covered by the genomic sequence assembly was designed using the Clone Manager Professional suite version 9 . This primer was extended on 500 ng of viral genomic DNA in 100 µL of 1 × PCR buffer, 1 × Q solution, and 5 U Taq and 0.2 U ProofStart Taq polymerase , 0.4 µmol/L primer D2 RII and 0.05 mmol/L each dNTP. The thermocycler was programmed for 35 cylces of 94 °C for 10 s, 57 °C for 60 s, and 68 °C for 3 min (with 95 °C for 2 min at the start and 72 °C for 10 min at the end of the program). This PCR reaction was purified with the QIAquick PCR Purification Kit, 25 µL thereof were incubated with terminal transferase  in a final volume of 50 µL of 1 × Tailing Buffer, 0.25 mmol/L CoClA nested PCR was next performed to recover the 5′ extended and dC-tailed product using primers D2 and the universal anchored primer AAP in a final volume of 100 µL as described above for D2 RII primer extension but with an extension temperature of 59 °C for 60 s (instead of 57 °C).This first nested PCR was diluted 1:50 and subjected to a second nested PCR in a final volume of 50 µL, without Q solution, primers GSPD2-R and AAP, with the same thermocycler program as in the first nested PCR. A fragment of approx. 700 bp was isolated and purified by agarose gel electrophoresis with the Qiagen Gel Extraction kit and sequenced with primers AAP and GSPD2-R. Primers were designed on this sequence as a diagnostic pair for amplification of 469 bp spanning the newly discovered recombination site (RS469).Bcl I, Nru I or ApaL I was performed with 3 µL of PCR product in 20 µL final volume and 0.5 µL of enzyme according to the manufacturer’s instructions.The long-PCR for amplification of the presumed left ITR of MVA-CR19 was performed with primers D2 RII and ITR-M to obtain 21,312 bp on MVA-CR19 and 9360 bp on wildtype MVA . Ultrathin sections (approximately 50-60 nm) were prepared using a Leica ultracut UCT  and contrasted with uranyl acetate followed by lead citrate. The ultrathin sections were examined in a Tecnai G2 Spirit BioTWIN transmission electron microscope  and digital images were acquired using a 2 k × 2 k side-mounted Veleta CCD camera .Single-cell cultures (not suspended aggregates) of 20 mL × 2 × 10et al. et al. Recombinant MVAs can be characterized by a set of PCRs that are designed to amplify across each of the six deletion sites  chorioallantois vaccinia Ankara virus (CVA)  sequence in the core of the genome and faces towards the left ITR. The expected amplification product with GenBank AY603355 as template is 9360 bp long and contains deletion site I.  A rearrangement of the right ITR to the position of the left ITR increases the amplification product from 9360 to 21,312 bp . Only MVA005R (C11R in vaccinia virus Copenhagen), MVA006L (C10L) and MVA008L (D7L) therein appear to be functional genes  in three contigs of 15,557, 89,342 and 40,737 bp.et al. All previous mutations could be confirmed and only one additional point mutation was discovered, V110A in MVA082L (L3L in vaccinia virus Copenhagen). We also sequenced L3L directly by conventional chromatography method in preparations of passage 2 MVA, passage 11 MVA and the plaque-purified MVA-CR19, and observed two notable differences to the previous three point mutations: there appeared to be no visibly mixed population of L3L genotypes in MVA-CR11, and there were no indications of the presence of L3L in the passage 2 preparation [The A9L mutation of the CR-genotype was already visible in passage 2 of non-plaque purified MVA, and all CR genotype mutations were overlapping with wildtype sequence in chromatograms of MVA-CR11, see Fig. et al. et al. et al. et al. et al. We previously observed a pronounced shift towards the novel strain in populations with mixtures of wildtype viruses and those that carry the MVA-CR mutations after repeated passage in suspension cultures . We did not observe differences in the early steps of morphogenesis Fig. C.et al. et al. et al. et al. Core wall formation and DNA condensation is one step towards intracellular infectious particles or mature viruses (MV). A3 contributes as a structural protein of the core and A9 may be involved to connect the core and MV envelope  now have three lipid layers and are further transported to the cell periphery. If the outmost viral envelope fuses with the plasma membrane, a cell-associated virion with two membranes is formed. In this configuration, the virion can either infect neighboring cells  or detach to carry the infectious units as extracellular enveloped viruses (EVs) to distant sites. The outer membrane of the EV helps to protect the MV against the adaptive immune system of the host and must be shed prior to infection of new host cells Moss . The EV et al. L3 is an essential protein for vaccinia virus replication (Upton Hph I site is deleted) that allowed a comparison of MVA-CR19 and MVA-CR19.GFP to sequences obtained from wildtype virus or viruses passaged on the fruit bat cell line. The Hph I site polymorphism complements sequencing and was detectable in wildtype and bat-cell passaged MVA viruses (MVA-RT20) but not in MVA-CR19  was used to quantify how much of the cell area is covered with GFP-expressing (infected) cells at two time points (24 and 40 h p.i.) and increasing MOI . Because correct MOI is an important parameter in this study, two separate preparations of MVA-WT.L3LCR.GFP (the L3L mutation of MVA-CR19 in the backbone of wildtype virus) were tested. As shown in Fig. et al. et al. et al. et al. et al. et al. Maintenance of transgene expression, replication properties and degree of attenuation depend on the genomic stability of viral vectors. Although virus isolates with deletions and rearrangements in the ITR  may interfere with the capacity of MVA to engage extracellular signal-regulated kinase 2 (ERK2) that causes NF-κB activation  in the left ITR may be compensated by the duplication of MVA184R (B16R) in the right ITR. Both proteins are antagonists for the proinflammatory IL-1β. However, because the two factors act by different mechanisms  has also been proposed previously to both increase reactogenicity and the safety profile of MVA  is also duplicated by the rearrangement. B5 is a structural protein involved in the intracellular wrapping with Golgi-derived membranes that lead to formation of extracellular virions  and F-box remaining in MVA. Viral ANK/F-box proteins can connect the cellular ubiquitin ligase with selected target proteins and are implicated in the regulation of host range and defense against innate immune responses Below is the link to the electronic supplementary material.


Many nations have established primary health care (PHC) organizations that conduct PHC planning for defined geographical areas. The Australian Government established Primary Health Networks (PHNs) in 2015 to develop and commission PHC strategies to address local needs. There has been little written about the capacity of such organizations for evidence-informed planning, and no tools have been developed to assess this capacity, despite their potential to contribute to a comprehensive effective and efficient PHC sector.We adapted the ORACLe tool, originally designed to examine evidence-informed policy-making capacity, to examine organizational capacity for evidence-informed planning in meso-level PHC organizations, using PHNs as an example. Semi-structured interviews were conducted with 14 participants from five PHNs, using the ORACLe tool, and scores assigned to responses, in seven domains of capacity.There was considerable variation between PHNs and capacity domains. Generally, higher capacity was demonstrated in regard to mechanisms which could inform planning through research, and support relationships with researchers. PHNs showed lower capacity for evaluating initiatives, tools and support for staff, and staff training.We critique the importance of weightings and scope of some capacity domains in the ORACLe tool. Despite this, with some minor modifications, we conclude the ORACLe tool can identify capacity strengths and limitations in meso-level PHC organizations. Well-targeted capacity development enables PHC organizations’ strategies to be better informed by evidence, for optimal impact on PHC and population health outcomes. The importance of using evidence to inform health policy and planning decisions is well recognized. These decisions are also influenced by practical, political and ideological factors. Much research has sought to identify key barriers to the ‘push’ and ‘pull’ of evidence for use in health policy-making , which cExtensive literature on the use of evidence in decision-making spans a broad range of disciplines, and health policy-making and clinical practice are areas that feature prominently. In between these realms of broad population-based approaches implemented by governments, and individual services delivered by health care professionals is the meso level of health planning, which involves devolved, relatively autonomous regional decision-making for a geographically defined population. While not strictly ‘policy-making’, it similarly involves making decisions and allocating public money to interventions designed to improve population health, amidst a complex range of competing influences. Devolved decision-making, as with that of ‘higher’ levels of government, is likely to be most useful and least harmful when informed by the systematic and transparent use of a range of different types of evidence , 15, yetMeso-level, regional primary health care organizations (PHCOs) feature in the health systems of numerous high-income countries, including the United Kingdom , New ZeaThis paper provides a reflective critique of the use of a slightly adapted version of the ORACLe tool to assess the organizational capacity of meso-level PHCOs for evidence-informed health planning critically, and presents the findings of applying the tool to Australian PHNs.This research extends on a larger National Health and Medical Research Council-funded mixed-methods research programme that ran from 2014 to 2018, and examined various aspects of Australian PHCOs : governance, health equity, comprehensive PHC approaches, population health planning, evidence use in planning and more. The research has employed a range of methods; however, the principal approach drawn on in this paper has been case studies of five PHNs. A purposive sample of PHNs were recruited, ensuring a range of metropolitan and rural/remote PHNs from different states/territories. Four of the PHNs had participated in earlier stages of the research, and another was recruited on the basis of similarities with a discontinuing PHN. Two of the five PHNs were in metropolitan areas, one was in a rural area, and two covered both metropolitan and rural areas. To ensure anonymity, participating PHNs are referred to as Metro North, Metro South, Rural North, Rural South, and Remote.This paper focusses on data drawn from 14 semi-structured interviews with employees from participating PHNs, conducted in 2018 specifically for this component of the research. The interviews also included a range of questions on other factors relevant to evidence-informed planning. Analysis of internal policy documents supplemented the interview analysis. Qualitative analysis of all interviews was conducted alongside ORACLe tool analysis.The ORACLe tool is an interview schedule of 23 questions, designed to examine organizational capacity for evidence-informed policy-making in health policy agencies. Documented processes to develop policy and plans that encourage or mandate the use of research (11.88%)Tools and programmes to assist leaders of the organization to actively support the use of research in policy and programme development (19.48%)Availability of programmes to provide staff with training in using evidence from research in policy and in maintaining these skills (20.53%)Availability of support and tools to help staff access and apply research findings (17.57%)Presence of systems/methods to generate new research evidence to inform the organization’s work (8.74%)Clear methods to allow adequate, evidence-informed evaluations of the organizations’ policies and programmes (10.96%)Mechanisms that help strengthen staff relationships with researchers 10.84%) [4% [8]The tool examines capacity across seven domains, each weighted for their relative importance:The ORACLe tool questions were incorporated into a semi-structured interview schedule that addressed a range of issues related to evidence-informed, equity-focussed health planning, including organizational capacity.In order to maintain the established validity of the tool, little change was made to the content and intent of the questions, other than to adapt some wording to reflect PHN planning, rather than government policy-making. For example, questions that originally specified ‘in the last 6 months’ were changed to ‘12 months’ to reflect the annual commissioning cycle of PHNs. In preparing the ORACLe-based interview schedule, the scoring guide was consulted to help ensure that further ‘probe’ questions were included so that sufficient detail could be obtained for scoring. Given that interviews were semi-structured, there was some variation as to the use of further probing questions, as necessary, and sometimes in the ordering of questions, in response to the ‘flow’ of the more conversational style of interviews. The interview schedule was piloted with two interviewees from a non-participating PHN.Participating PHNs were invited to nominate interview participants, representing three different levels of involvement in planning and programme development—CEO (or deputy), and a senior manager and staff member involved in planning. One PHN nominated only two participants because they only had a small team to draw from. None of the invited interviewees declined. All interviewees gave informed consent to participate in the research, and none dropped out. Of the 14 interviews, 13 were conducted face-to-face at the respective PHN, and one was conducted via telephone. Interview duration ranged from approximately 60 to 80 min. There were no non-participants present in interviews. Interviews were conducted between May and September 2018, and there were no repeat interviews. Interviews were conducted by AW, a female PhD candidate who has experience in qualitative interviewing, and has worked for PHCOs in planning. Two of the interviewees had prior professional peer interactions with the interviewer, and the rest had only a preliminary introduction to the research and interviewer prior to participation. At the beginning of each interview, the interviewer introduced herself and provided a summary of her experience in PHCOs and the research aims. All interviews were digitally recorded and professionally transcribed. Field notes were also made during and after each interview. All interviewees were offered the opportunity to review their transcript prior to analysis.Ethics approval was granted by the Flinders University Social and Behavioural Research Ethics Committee , and all participants gave informed consent to participate.Transcripts were coded using NVivo qualitative analysis software , using a coding framework of key research themes drawn from a conceptual framework of evidence-informed health policy-making , and theThe original application of the ORACLe tool involved only one CEO interview per organization, which made for simple allocation of a score based on the responses from one person. As recognized in the open peer review of the ORACLe paper , and by Once coded to relevant nodes, ORACLe data were extracted from NVivo into a MS Word table, with responses organized by question for each PHN. The approach to scoring each question involved first consulting the scoring guide for the respective question , and theWhile the original ORACLe tool involved scores of only whole numbers , this application of the tool allowed for increments of 1.5 and 2.5 where the response was greater than the lower score, but did not qualify for the higher score, according to the scoring guide. Intermediate scores have been used in other applications of the tool . IntermeOnce consensus scores had been assigned for all questions and PHNs, scores were entered into a MS Excel spreadsheet. Total and average question consensus scores within capacity domains were calculated. Total weighted scores for each PHN were also calculated using the conditional logit model outlined in Additional File 1 of the ORACLe paper .Thirty internal policy or guidance documents were sourced. Twenty-six were provided by PHNs and four were downloaded from their websites. These documents were examined for documented planning processes/procedures. This evidence was then triangulated with interview responses regarding documented processes, from Domain 1 of the ORACLe tool. While this aspect was not specified in the original ORACLe procedure, we collected these data to add rigour.Before using the ORACLe tool in this research, a theoretical critique was conducted to examine its alignment with key theory in the international literature on evidence-informed health policy-making and to consider its appropriateness for application to PHCOs. This critique is examined in the “A practical critique of the ORACLe was based on the reflections of the first author’s experience in piloting the tool, using it in interviews and subsequently coding data and assigning scores. This drew on the field notes taken during and after interviews, and a methodology journal kept during the process of data coding and analysis.Application of the ORACLe tool identified variation between PHNs and between capacity domains. On the basis of unweighted, average domain scores, no one PHN consistently scored higher or lower than others Fig. .Fig. 1PHPHNs generally demonstrated moderate to high capacity, based on average domain scores ranging from 1.9 to 2.8 (out of 3).Strongest capacity was demonstrated in generating new research, through frequent, recent internal research such as focus groups, surveys, and data analysis, or through externally commissioned research projects (Domain 5).Several PHNs demonstrated moderately strong capacity through the reported existence of documented processes that provide detailed guidance on planning/programme development, and explicitly encourage or require evidence use (Domain 1). PHNs also demonstrated moderate to strong capacity in terms of mechanisms for relationships with researchers, by virtue of having both formal and informal relationships and participation at conferences, and to lesser extents, researcher participation in advisory groups, and joint appointments with research organizations (Domain 7).Moderate capacity was demonstrated in relation to leadership supports for the use of evidence, in that for most PHNs, leaders’ position descriptions explicitly covered expertise in use of research, and generally there was relevant professional development available for leaders. There was variation in the extent to which leaders reported mentioning research/evidence in their internal communications (Domain 2).There were three capacity domains for which average scores sat below the midpoint of 2, indicating relatively lower capacity. Training for staff was generally available and considered in performance management, although mainly not specifically regarding evidence use, and was ad hoc (Domain 3). PHNs’ supports for accessing and applying research evidence varied considerably. Staff with such expertise were relatively common, but there was variation in the extent to which relevant research was internally disseminated, or there were subscriptions to research journals and databases, or documented methods to commission reviews. There was generally low capacity in terms of knowledge management systems, libraries, reference management software, and particularly resources to guide the use of research evidence (Domain 4). While PHNs generally encouraged evaluation to be built into programme development, either implicitly or explicitly, capacity was lower in terms of documented, evidence-informed processes for conducting evaluation (Domain 6).PHNs’ scores tended to be lower in the domains with high importance weightings  and higher in the lower weighted domains .When recommended weightings were applied and domain scores totalled , there wKey strengths and limitations of the usability and value of the ORACLe tool, in meso-level PHC planning are outlined below.“Probably [manager] would be better able to inform you as to what happens with that stuff [various evidence sources/materials] when I send it through to them. I have to say I don’t know what happens to an awful lot of the stuff that I push down” .The decision to allocate scores on the basis of several interviews including the CEO was warranted, as there were indications in several instances where the CEO was not best placed to respond, for example:An assessment of all interviews and questions suggests there were six responses for which there was insufficient data to allocate a score for one of the respondents. In these cases, it was valuable to have other responses to refer to. There was complete agreement between responses from all participants within a PHN on around half of the questions. There was also a degree of disagreement, which partly reflected the varying detail of responses, as well as the different perspectives and knowledge of interviewees.Examination of PHN internal documents supported the favourable interview data regarding documented processes that encourage or require evidence use (Domain 1). The various documents examined tended to include broad guidelines for planning, such as overviews of the general commissioning cycle, and very few included specific procedures for programme planning, as the ORACLe tool sought. The documents did not include any prompts or mechanisms by which to critique the appropriateness or transferability of evidence for the issue under consideration. The most relevant documents were comprehensive commissioning toolkits or manuals (three PHNs), and templates for activity or programme plans (three PHNs). There was much to indicate that use of evidence was encouraged, but only a few examples of an evidence base or rationale being documented.When there were several criteria involved in scoring a response, it was more difficult to allocate a score, and the ability to allocate an intermediate score (1.5 or 2.5) was helpful. For example, four PHNs were allocated a score of 2.5 on the question of whether the PHN has documented processes for how plans/programmes should be developed . To achieve the full score of 3, there had to be detailed, organization-specific written guidance. In each of these PHNs interviewees were not confident in the degree of detail, so while there were documents, developed by that PHN, the full score was not warranted because of documentation lacking detail.Our inclusion of additional ‘probe’ questions into the interview schedule proved to be essential to obtain sufficient detail for allocating scores to the responses.The existence of documented processes that encourage the use of evidence is a positive and important aspect of organizational capacity. However, the value of such documents relies on how well the guidance is applied or the processes adhered to. Despite high scores in this domain (1), it was reported by interviewees (from a high scoring PHN) that documented processes were not consistently followed because they were too detailed, and their use was not actively encouraged. Some inconsistency between interview responses within this PHN suggests that while documented processes exist (and a high score is warranted), they are not consistently used.Our application of the ORACLe tool raised some general issues concerning its structure, flow and wording. For example, there was some overlap of questions and concepts between different domains, which made some aspects of the interview seem repetitious and disjointed. For example, there were two nonconsecutive questions about training for leaders (Domain 2) and training for staff (Domain 3), yet interviewees tended to talk about them all together. However, using NVivo, responses could be coded (and scored) to the relevant domain regardless of which question was being answered.referring to research/evidence in their internal communications, whereas four interviewees responded about using evidence in planning/programme development. Several questions may have benefitted from a preamble statement or preliminary question to clarify the focus.Some questions were commonly misinterpreted by interviewees, suggesting they may need to be worded differently. For example, Question 5 (Domain 2) asked about leaders access to a library may have been appropriate. Some interviewees’ responses implied that they perceived certain questions as unrealistic: “Um, goodness me…” .Some questions, as worded, were not appropriate to the PHCO context. For example, Question 11.3 asked about having a library, which while reasonable in a large government department, is unlikely in a small organization. Subtly changing the question to ask about In some domains, the scores for certain questions were dependent on responses to other questions. For example, if 1 was scored for Question 17 , 1 was the only possible score for Question 18  which potentially skewed results in this domain, which had the lowest average domain score, and considerable range (1.33–2.33).In Questions 20 and 21 , the score was based on the number of relationships, which seems a somewhat crude way to measure engagement with researchers, and does not examine mechanisms to facilitate relationships.Deeper probing for detail may have been beneficial in some questions. Many of the questions, as written in the original interview schedule, did not include sufficient probes to obtain the level of detail required by the scoring guide. Better interviewer technique to pursue the appropriate level of detail may also have helped; however, this can be challenging in a semi-structured interview where the order of questions can vary. Maintaining rapport with the interviewee and not being too assertive in asking critical questions is also an important consideration.“Evidence yes. Research hmm?!” ,“Aah, I don’t know if it’s the use of research, it’s about being evidence-based” A criticism of the original tool was the use of the term ‘research’, and in practice, wording was sometimes adapted to reflect the broader concept of ‘evidence’. Some interview participants’ responses also explicitly distinguished between ‘research’ and ‘evidence’:Although not recommended in the original ORACLe tool, qualitative analysis of ORACLe and additional interview data added to the ORACLe results provided a richer understanding of planning processes and influences on capacity. This analysis also identified ongoing strategies to improve aspects of capacity that did not yet strictly qualify the organization for a higher score. For example, one PHN was in the process of developing an evaluation framework, but did not yet have a documented guide for evaluating programmes. Qualitative analysis also identified a broad range of relationships between PHNs and other organizations, and various direct and indirect influences these have in evidence-informed planning. For example, strong relationships with state/territory government health departments enabled access to hospital and other data, which were an important evidence source for PHNs.Our application of the ORACLe tool demonstrated it could be used to identify strengths and weaknesses in PHCOs’ organizational capacity for evidence-informed planning. This section will discuss the utility of the tool for this purpose, explain some modifications we made, and explore criticisms we identified, reflecting on some key theoretical considerations regarding capacity elements and hierarchy.Our research indicates that the ORACLe tool, while developed for state or national health policy-making, can be applied to examine organizational capacity for evidence-informed decision-making in meso-level health planning organizations. Within organizations it can help to identify certain aspects of capacity that may benefit from further development. It also offers value in potentially identifying common capacity limitations across like organizations, which may be addressed by broader capacity development strategies.Our findings suggest that PHNs’ capacity, based on adjusted weighted domain scores, is comparable, if not slightly stronger than that of government policy agencies . PHNs geWhile this research indicates that ORACLe is valuable in examining capacity strengths and weaknesses, it does not allow for a qualitative understanding of why capacity might be limited, or progress in developing capacity. We found that qualitative analysis can complement ORACLe findings in these respects, as well as allowing a broader exploration of related themes.We also undertook some document analysis to complement the ORACLe tool which was helpful in corroborating Domain 1 findings. Further document analysis (e.g. position descriptions) may have helped to validate responses to other relevant questions. While the ORACLe tool examines the existence and content of documents, it does not allow for examination of the extent to which documents are used or reflect practice. It also does not go to the detail of examining if/how evidence is appraised and applied. A more comprehensive ‘audit’-type process may yield more valid findings about organizational capacity and process.This research was strengthened by administering the ORACLe tool with several participants from each organization, instead of only the CEO. It helped to overcome some paucity of data, either through insufficient probing or interviewee knowledge gaps. If detail was lacking from one interview, it could be obtained from another. It did complicate the scoring in some instances, where responses differed and judgement was required; however, we consider this made for a more accurate result. We also found that inconsistent responses within an organization can indicate issues with internal policy implementation, in that documented guidance, while it existed, was not necessarily followed.Our inclusion of additional ‘probe’ questions was essential to obtain sufficient detail to allocate scores to responses. That these additional questions are not necessarily consistent in other applications of the tool is a further reason for exercising caution when comparing PHN capacity findings from this research, with capacity studies of other organizations.Our experience highlighted one concern with the ORACLe tool in the use of the term ‘research’, which reflects, and potentially constrains responses to a narrow conception of evidence. While much of the evidence-informed policy literature focusses on research evidence generated by academics and published in peer-reviewed journals, it is frequently recognized that decision-makers’ conceptions of evidence will differ from those of academic researchers . A wide We suggest that the order and interpretation of questions may benefit from some rearranging, or a preamble, to clarify the focus of some questions. Some interdependence between questions for allocating scores was identified and may need to be addressed, such as the questions regarding evaluation processes, as outlined above. Some questions may also benefit from minor wording changes, to make them more applicable to small–medium, meso-level organizations with annual planning cycles, rather than large government agencies.Domain 7 of the ORACLe tool examines capacity regarding staff relationships with researchers. Our broader qualitative analysis identified a range of important relationships in addition to those with researchers, indicating that this domain of the ORACLe tool is potentially too narrow. While this domain of the ORACLe tool is based on a wealth of sound evidence that relationships with researchers facilitate evidence use , an expaprocess of evidence-informed policy-making as opposed to the outcomes of policy-making, against the principles of appropriateness, transparency, accountability and contestability.Governance is recognized as an element of organizational capacity for evidence-informed policy/planning  yet was While Domain 1 partly addresses governance by examining the existence and detail of ‘documented processes’, the assessment of governance capacity for evidence-informed decision-making in the ORACLe could be strengthened by more detailed examination of systems to embed key governance principles. In particular, appraisal of the appropriateness or transferability of evidence is important in meso-level regional health planning, to ensure that the strategies for which evidence is considered are appropriate for the region or community in question. A hypothetical example might be where a strategy to enhance the cultural safety of services for Aboriginal and Torres Strait Islander people has been favourably evaluated in X region—can it then be confidently assumed that the same strategy would be culturally safe in Y region ?A further issue with the ORACLe tool relates to the importance weightings of capacity domains. Domain weightings  structures and systems. This echoes a key challenge in public health: the persistent adoption of individualistic, behavioural health promotion strategies, rather than more effective population-based, systemic approaches to improve public health . Green aIt is recognized that development of systems and structures in an organization is more difficult and time consuming than developing individual skills or tools. Potter and Brough  argue thAn alternate weighting system, informed by theory, and/or appropriate to the meso-level planning context would likely produce a more valid assessment of PHNs’ capacity for evidence-informed health planning. As it stands, it would not be valid to use the ORACLe tool to compare organizations from different decision-making contexts. However, comparison of unweighted domain scores can be useful to indicate capacity shortcomings within an organization, and also between similar organizations. We would argue against weighting the different domains as to their overall importance, and instead focus on the applicability of the tool to identify specific areas for capacity development within organizations.The developers of the ORACLe tool recommend that data coding and scoring not be done by the same person who conducts interviews . One of Another limitation of this research was that there was no participant checking of the ORACLe scores and findings; however, we feel this was mitigated by drawing on responses from multiple interviewees in assigning scores.The ORACLe tool was developed and intended to be used alongside the SAGE tool  which exThe approach taken in this research did not allow for direct comparison between the ‘standard’ ORACLe methodology as described by its creators  and the Competence in policy and planning decision-making is just as important at the meso level as it is at higher levels of government, and capacity for evidence-informed decision-making is a key aspect of such competence.Through this research, we have demonstrated that the ORACLe tool can be useful to examine aspects of organizational capacity for evidence-informed planning in meso-level PHC organizations. Our application of ORACLe has identified some opportunities to refine or complement the tool, which are outlined in Box Box 1Recommendations to enhance the ORACLe tool:Collect data from several individuals within the same organization, to improve the validity of findings.Include additional questions to probe for detail.Undertake qualitative analysis of broader interview data to provide deeper understanding of capacity limitations.Examine relationships and networks more broadly than those with researchers.Include additional interview questions and document analysis to examine capacity for leadership and governance of the decision-making process.Analyse internal organizational documents to triangulate interview findings.Look at a ‘higher’ level of capacity—at the ‘systems and structures’ rather than tools and skills, and if weightings are to be used, they should reflect this.To improve the appropriateness of the tool for meso-level planning:Adopt and make explicit a broad definition of ‘evidence’ that includes material from non-academic sources, rather than ‘research’, and use this terminology consistently.Include some examination of mechanisms to assess the appropriateness of evidence to the context in question.access to a library?).Adjust wording slightly to make some questions more relevant (for example, is there


To examine the quality of evidence used to inform health policies. Policies on peripheral intravenous cannulas were used as exemplars.An organizational case study design was used, using the STROBE reporting guidelines.Policy guidelines were sourced between June and September 2018 from health departments in Australia. Seven documents were compared regarding intravenous cannula dwell times and blood collection use. Evidence used in the documents was critiqued using assessment guideline from the Oxford Centre for Evidence Based Medicine.Large variations exist between policies regarding blood sampling and dwell time. Evidence used a variety of sources. Few references received an A evidence rating and policies differed in their interpretation of evidence. The term EBP is associated with the work of Archie Cochrane and his belief that clinicians should use only the most effective and proven healthcare procedures available  is medical devices that are inserted into and remain in a person's peripheral vein to allow continued venous access , New South Wales (NSW), Northern Territory (NT), Queensland (QLD), Tasmania (Tas), Victoria (Vic) and Western Australia (WA). During data collection, it was found that South Australia (SA) did not have an overarching policy for the use of PIVC, but that polices were set by several local health networks. Due to the inability to access several the local health network policies, it was decided to exclude SA from this study.The policies were assessed according to firstly, how well the policy covered the procedure of PIVC blood draws and secondly, PIVC dwell time. Data were extracted from the documents including the year the policy was published, the title of the document and whether the policy included reference to the practice of sampling of blood from a PIVC. Also extracted were details of whether blood could be drawn from a PIVC on insertion, could blood be routinely drawn from a PIVC, could a PIVC be inserted for the purpose of blood collection, was there a procedure on how to draw blood from a PIVC, were there any exceptions and did the policy outline when a PIVC should be removed or replaced.The evidence and type of evidence that was used to support these guidelines were also extracted. To determine the type of evidence being used to justify positions taken in the guidelines, the reference lists and bibliographies of the guidelines were analysed. Data were collected relating to evidence type, where the evidence was taken from, who created the evidence and how recently it was published.Using assessment guidelines set out by the Oxford Centre for Evidence Based Medicine (OCEBM) , the cho4Data extracted from the policy documents identified large variations between policy documents in the information provided. Table The type of evidence used by the different documents also varied  of specific practices prior to guideline publication. Policymakers may wish to consider providing grades of evidence to support specific policies, allowing for more open discussion over where new evidence is required.Where there is clear evidence for a clinical policy, such as dwell time for PIVCs, decision‐makers appear to be ignoring the evidence. State governments should consider employing people with academic skills and clinical experience in related professions in advisory roles prior to publication of health policy documents to review the evidence.Involving clinicians and clinically trained academics in policy creation could assist with reducing the divide between policy and practice, as well as providing a conduit for communicating guidelines to clinical professionals. Future research could consider the capacity of clinical staff to engage with varying levels of evidence and the upskilling required to assist clinicians to think critically about knowledge translation.6The use of high‐quality evidence in the creation of health policy is important to ensure that the policy results in positive health outcomes. Health policy guidelines for the use of PIVC in blood sampling in Australia are inconsistent and inconsistently supported by evidence. While it may not always be feasible for decision‐makers to base policies on evidence, they should be at the least informed by evidence. Barriers to evidence‐based policy can include lack of available or consistent evidence and lack the skills of skills by decision‐makers to understand academic level clinical information. Governments should be at the forefront for using research‐based evidence where available instead of informing themselves through the expert opinions and individual contacts.The authors declare that they have no conflicts of interest associated with this study.AJ and EJ: Study design. AJ: Data extraction. EJ, AJ, HD and LC: Methodology. AJ and LC: Data analysis. AJ, HD, LC and EJ all contributed towards editing and all authors approved the final manuscript.Supplementary MaterialClick here for additional data file.


The purpose of this study was to comprehensively assess the heterogeneity of procedures in the diagnostic guidelines for acute pancreatitis and to identify gaps limiting knowledge in diagnosing this disease.A systematic search of a number of databases was performed to determine the guidelines for the diagnosis of acute pancreatitis in patients with severe pancreatitis. The guidelines for the diagnosis of severe pancreatitis were evaluated by AGREE II. The Measurement Scale of Rate of Agreement (MSRA) was used to assess the guidelines (2015–2020) and extract evidence supporting these recommendations for analysis.Seven diagnostic guidelines for acute pancreatitis were included. Only the 2019 WSES Guidelines for the Management of Severe Acute Pancreatitis and the Japanese Guidelines for the Management of Acute Pancreatitis: Japanese Guidelines 2015 had a total score of more than 60%, which is worthy of clinical recommendation. The average scores of the Scope and Purpose domain and the Clarity and Expression domain were the highest at 71.62% and 75.59%, respectively, while the average score of the Applicability area was the lowest at 16.67%. The included guidelines were further analyzed to determine the heterogeneity of the diagnosis of acute pancreatitis. The main reasons for the heterogeneity were the citation of low-quality evidence, the presence of far too many indicators for the classification of acute pancreatitis, unclear depictions of the standard, and poorly comprehensive recommendations for the diagnosis of the aetiology in the primary diagnosis of acute pancreatitis, the severity classification, the aetiological diagnosis, and the diagnosis of comorbidities.The quality of different diagnostic guidelines for severe pancreatitis is uneven. The recommendations are largely based on low-quality evidence, and the guidelines still have much room for improvement to reach a high level of quality. The diagnostic procedures for acute pancreatitis vary widely in different guidelines. There are large differences between them, and resolving the abovementioned reasons would be a very wise choice for guideline developers to revise and upgrade the guidelines in the future.The online version contains supplementary material available at 10.1186/s13643-020-01559-4. Acute pancreatitis is an inflammatory process that can lead to local and systemic inflammatory reactions. Globally, the incidence of acute pancreatitis is increasing and is expected to further increase medical burden . AlthougA standardized diagnosis of acute pancreatitis is essential for the early treatment of the disease. In recent years, many academic organizations and experts in related fields have developed guidelines for the diagnosis of acute pancreatitis –8. The dTherefore, this study intends to adopt the The Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool to comprehensively evaluate the current guidelines for the diagnosis of acute pancreatitis, comb the diagnostic procedures for acute pancreatitis, and analyse the differences among the recommendations and supporting evidence for the diagnostic steps for acute pancreatitis in the current guidelines so that future guideline developers can make meaningful improvements and updates.This study was developed through international collaboration and discussion with a panel of experts composed of managers engaged in emergency surgery and patients with severe acute pancreatitis.Diagnostic studies were searched in the PubMed, Ovid, Web of Science, Science Direct, CNKI, and Wanfang databases (2015–2020). Keywords include the following: “Severe Pancreatitis,” “Pancreatitis,” “SAP,” “Acute Pancreatitis”; “Guidelines,” “Recommendations,” “Consensus.” “Statements.” The terms were first retrieved individually and jointly. Then, the set operator “and” was used to combine the results with studies identified through the term “diagnosis.” References that were included in the guidelines were manually retrieved to identify potential additional guidelines.Inclusion criteria were as follows: (1) guidelines for the diagnosis of severe pancreatitis; (2) a complete guideline text, published in English or Chinese; and (3) the latest version of a set of guidelines that has been updated multiple times.The exclusion criteria were as follows: (1) reviews and comments on the diagnosis of severe pancreatitis; (2) interpretations of the guidelines for severe pancreatitis; (3) repeated publication guidelines; (4) non-English- and non-Chinese language guidelines; and (5) guidelines whose full contents could not be retrieved.We used the latest version of the AGREE II tool to evaluate each set of acute pancreatitis guidelines that met our inclusion criteria . AccordiIn this study, four reviewers  scored each guideline for the diagnosis of severe pancreatitis based on the AGREE II user manual, independently evaluated the search results to determine whether to include or exclude references, and extracted the general characteristics of each guideline. If there were differences, they were resolved through consultation and mediation or consultation with other expert reviewers. Among these four reviewers, Y.M.Y., H.B.F., Y.Q.D., and D.L.S. are responsible for the clinical diagnosis and treatment of severe pancreatitis. In addition, D.L.S. and T.Y. have extensive experience in applying AGREE II. All authors have undergone standardized training on the AGREE II tools.The AGREE II user manual defines each item and assists the user in scoring the guidelines. The items are scored on a scale ranging from 1 (no item) to 7 . The scores are summed and then normalized to the percentage of the maximum possible score, and the domain score is calculated. The overall assessment includes whether the guideline is recommended for clinical practice. In general, the AGREE II group divides the overall assessment into three categories: recommended, revised recommended, and not recommended. Consensus was reached based on the results of 23 evaluations and the reviewers’ overall judgment. A guide was classified as “Recommended” if it achieved a total score of > 60%, “Revised recommended” if it achieved a total score of 30–60%, and “Not recommended” if it achieved a total score of < 30%.To analyse the heterogeneity of the recommendations and the evidence for the diagnosis of severe pancreatitis among the guidelines, we used the Oxford Centre for Evidence-based Medicine-Levels of Evidence (OCEBM) evidence grading system  to reclaP < 0.05 was considered statistically significant. SPSS version 17.0  was used to conduct statistical analysis.We used a descriptive statistical analysis method to calculate the standard score for each domain, expressed as a percentage, and also list the median and range for each domain score. In addition, the distribution of the number of recommendations and the level of evidence is also expressed as a percentage. Two-way ANOVA was used to test the consistency between each reviewer’s scores using the within-group correlation effects of single-factor raters. According to previous studies, a degree of consistency between 0.01 and 0.20 is considered slight, 0.21–0.40 is considered moderate, 0.41–0.60 is considered moderate, 0.6–0.80 is considered substantial, and 0.81–1.00 is considered very good . P < 0.0A total of 122 related studies were retrieved, and some guidelines or consensuses were excluded for a variety of reasons. Finally, only seven guidelines or consensuses were included  , a validFour guidelines , 3, 7, 8Five guidelines –4, 7, 8 Seven guidelines –8 recommFive guidelines –5, 8 recCECT is recommended as a reference indicator for the diagnosis of severe pancreatitis in seven guidelines, four of which make explicit recommendations –6, 8, anCompared with other disease-specific guidelines, the establishment of a diagnostic guideline for severe pancreatitis may be a more complex issue; these guidelines are a global concern managed by a variety of disciplines. This study found that in the initial diagnosis of acute pancreatitis, severity classification, aetiology diagnosis, and comorbidity diagnosis, there are irrational references to evidence, many indicators for the classification of acute pancreatitis, unclear standard descriptions, and poorly comprehensive recommendations for the determination of the aetiology of acute pancreatitis. The quality evaluation of the included guidelines with AGREE II demonstrated that the quality of the diagnostic guidelines was uneven. Even if there was a high degree of heterogeneity in different domains for the same guideline, the scores of only two sets of guidelines were high, indicating that they could be recommended for clinical use.Although most scholars do not believe that the diagnosis of acute pancreatitis should follow a certain procedure, clear procedures would help guide users and readers to better understand and study the diagnosis of acute pancreatitis . One setOf the seven guidelines included, the similarity of the recommendations for abdominal pain, biochemical evidence, and imaging characteristics as criteria for the primary diagnosis of acute pancreatitis Table  was highI2 = 93.2), so the results are unstable.(1) There are many recommended grades for the severity of acute pancreatitis: of the seven guidelines included, a total of 8 grades are available for reference. The Atlanta classification, C-reactive protein level, and APACHE-II score are recommended by more than 3 guidelines, while other indicators are only recommended or mentioned in 1–3 guidelines. This is related to the lack of high-quality studies comparing the pros and cons of these acute pancreatitis severity indicators. 2) The clinical outcome indicators used in the guidelines to support the evidence of recommendation are significantly different. For example, in guideline  The clin, and theObviously, in the future, international organizations are expected to establish a global consensus on the severity classification of pancreatitis, so as to end the current confusion.(1) The recommendations for the diagnosis of the aetiology are not comprehensive; it is important to identify the cause of acute pancreatitis for its early treatment. The main cause of acute pancreatitis is gallbladder or bile duct stones, followed by alcohol use and smoking, other rare drug use-related causes, and factors such as hypercalcemia, hypertriglyceridemia, surgery, and trauma , 17. TheFor acute pancreatitis comorbidities such as pancreatic necrosis, peritoneal effusion, pseudocyst, and wall necrosis (WoN), the 7 included guidelines recommend the selection of CECT for a clearer diagnosis. If the effusion or necrotic tissue is considered to be infected, all guidelines recommend fine needle aspiration (FNA) to confirm the diagnosis of bacterial culture. Although the evidence of the recommendations for the diagnosis of acute pancreatitis comorbidities lacks high-quality studies, the evidence provided by the guidelines is very different. For example, for the CECT diagnosis of pancreatic necrosis, the evidence of three guidelines –6 is a rIn summary, this study puts forward suggestions for improving the procedures in the diagnostic guidelines for acute pancreatitis: (1) It is recommended that in the future guidelines, the concept of diagnostic procedures should be introduced, which should be standardized to logically help guide users to diagnose acute pancreatitis more clearly; (2) greater attention should be paid to the issue of evidence citation in the diagnosis of acute pancreatitis, with a focus on a comprehensive and systematic retrieval of existing evidence and fair citations; (3) the graded indicators of acute pancreatitis should be clearly described so that readers and clinical practitioners can use the guidelines accurately; and (4) the recommendations for the diagnosis of acute pancreatitis generally lack high-quality research support, and it is recommended that research teams participating in the development of guidelines pay attention to and carry out corresponding high-quality research.We need to make improvements in the following areas. Stakeholder engagement reflects how well the guideline represents the views of its intended users, including patients. Implementation of the guideline requires the contribution and expertise of a multidisciplinary medical team, including patients. As guidelines develop, patients’ perspectives, expectations, and preferences for healthcare become increasingly important. However, no guidelines have yet provided detailed information on the involvement of patients or their representatives. The rigor of guideline development is the most critical area and will affect the confidence of the implementation of the guidelines . UnsysteThe low applicability scores in the applied areas indicate that guideline developers have not paid enough attention to potential obstacles affecting the actual implementation of the recommendations . PrelimiThe results of our research required the careful re-evaluation of existing guidelines and improvements in the quality of formulation and reporting in practice. First, in addition to being familiar with the guideline development standards, instrument developers should apply these tools and, wherever possible, describe them in the guidelines . Second,In general, our research has some advantages. First, our authors come from different backgrounds, including clinical experts and methodologists with rich experience in the evaluation of clinical guidelines, which improves the reliability of our findings. Second, the different areas have been appropriately weighed to arrive at an overall assessment and recommendations. Nonetheless, our research has several limitations. The exclusion of guidelines published in languages other than English and Chinese or in other formats  may lead to an underrepresentation of guidelines from less developed countries. Second, the AGREE II tool focuses on guideline development methods and reporting transparency but cannot assess the potential impact of recommendations on patient outcomes , 29. In The diagnostic procedures in all guidelines for acute pancreatitis have unreasonable references to evidence, there are many grades of acute pancreatitis, the standard description is not sufficiently clear, and the recommendations for the diagnosis of the aetiology are not sufficiently comprehensive. The quality of the guidelines for the diagnosis of acute pancreatitis is uneven, and there is much room for improvement, especially in the areas of applicability, stakeholder involvement, and editorial independence. Acute pancreatitis guidelines should develop recommendations for high-quality evidence while minimizing external error with convincing methodological rigor and transparency.Additional file 1: Table S1. Levels of evidence based on the Oxford Centre for Evidence-Based Medicine


Carotid sinus syndrome has been associated with transient loss of consciousness for millennia, and while steeped in cardiovascular lore, there is little in the way of solid evidence to guide its main treatment modality, permanent cardiac pacing. This article reviews the history of the condition in the context of its contemporary understanding before examining three key concepts in the consideration of what constitutes a manageable disease: first, is there a pathophysiologic rationale for the disease (in this case carotid sinus syndrome)? Second, is there a good diagnostic test that will identify it reliably? And finally, is there a convincingly evidence-based treatment for the disease? Relevant literature is reviewed, and recommendations made in how we view pacing in the context of this intriguingly opaque condition. The Persian Muslim father of modern medicine Avicenna , Caleb Hillier Parry , made thcompression of the carotid artery, as they may be produced without obliteration of the calibre of the artery; or vice versa the course of the blood may be completely interrupted in the artery without producing any of the symptoms enumerated,”   . The 53-tivité”) *. Repeatollow-up .** “…as syncope occurred for more than 15 sec. following discontinuation of pressure, I auscultated attentively over the praecordium: absolute silence. Finally, generalised epileptiform convulsions ensued. Then, abruptly, the heart began to beat with an accelerated rhythm, at around 120 beats per minute, with initial frequent extrasystoles interrupting the succession to normal sinus rhythm”] vialitarius –42 and blitarius  42–45)..via neurlitarius , 46, andlitarius , 43, Furlitarius , 48. Altlitarius  and latelitarius , workinglitarius , 50. Thelitarius . HoweverOne study showed hyperphosphorylated tau accumulation baroreflex-associated neurons in a controlled neuroanatomical study of 12 patients with CSH compared to 14 controls , so therThis “physiologic” denervation hypersensitivity then causes the overshoot bradycardia and hypotension following carotid sinus stimulation that is clinical CSH . We testThere is thus a small but inconclusive evidence base to suggest neuroanatomical abnormalities as the underpinnings of CSS. What about a more functional disorder analogous to psychiatric disease in the absence of overt brain pathology? One possibility is disordered cerebral autoregulation, a candidate catalyst for the conversion of asymptomatic CSH to CSS, with linked studies by Leftheriotis et al. , 59 showIf cerebral autoregulation abnormalities are the ultimate expression of the cause of symptoms in CSS, a strong candidate for the mediating mechanism would be the same underlying process, autonomic dysfunction. Morley and Sutton found abnormal baroreflex sensitivity in CSS and sick sinus syndromes as measured by the phenylephrine pressor test . Almost Consensus guidelines state that the current standard diagnostic test for CSS is 10 s of bilateral, sequential, and longitudinal CSM, right then left (as the hypersensitive response is more likely to occur on the right), during electrocardiographic and (preferably beat-to-beat) blood pressure monitoring , 31, 65.The short answer to this question is no, if the standard of evidence required is that of the randomized controlled trial (RCT). Two recent high-quality reviews of pacing intervention in CSS, where syncope is the presenting complaint, found no high-quality evidence to support pacing as a treatment of choice , 32 despThe truth is rarely pure, and never simple”“Truth, in the sphere of day-to-day existence as much as in classic 19th century comedy, is seldom absolute. As the reader will be aware from the discussion so far, Wilde's pithy observation on the nature of truth has a particular resonance in attempting to answer the question posed by the title of this paper. Our putative disease, CSS, on balance from the small number studies and patients involved in trying to understand the physiologic bedrock of this elusive condition, appears to have some basis in autonomic dysfunction. What is less apparent is whether this represents a disease in need of management (at least in the sense of the word as here defined) or whether this is part of the autonomic spectrum of normal aging. On balance, the former seems more likely, although a definitive answer is not possible from the data so far available, with the distinction between CSH and CSS proving particularly difficult from a pathophysiologic perspective. To further make the waters muddy, there is considerable observational evidence to suggest that there are large numbers of older people who have CSH in the absence of symptoms. Kerr et al. systematically evaluated a random sample of community-dwelling elders and found that 39% had CSH overall, with a cardioinhibitory response in 24%—in the absence of any culpable symptoms . Older sMore troublesome is the changing face of the technique and criteria used to diagnose this apparent disease, morphing from “pressure” of up to 30 s to the current 10 s of longitudinal massage over the course of the last eight or nine decades. Current guidance, on very sound physiologic principles, defines the cutoff for CSS diagnosis as 6 s of asystole, ignoring the troublesome fact that many of the intervention studies since the 1980s used the 3 s criterion , 30, 73 The answer to the question posed therefore is … probably. There is some evidence of disordered physiology in need of a remedy to treat the symptoms, although the test to diagnose is not a good one and the evidence supporting the intervention is arguably weaker than the strength of the recommendations for its use. Without a doubt, much further work is needed, with more detailed work on pathophysiology to guide treatment strategies, a better diagnostic test, and more clear phenotyping of symptom presentation that then aligns with potential pacing intervention. In addition, given the differences in test performance and interpretation as well as evidence level recommendations, it would be useful to develop world-wide consensus on the diagnosis and management of CSS . Newer pBox 1Patient characteristics*Age 40 or over, presenting with syncopeNo recent stroke, transient ischaemic attack or myorcardial infarction and no significant carotid stenosis §Mixed CSS: asystole with significant vasodepressor response.ManagementModification of culpable medication where feasibleDual chamber pacemaker implantation may be indicated for cardioinhibitory or mixed CSS sub-typesGuidance*While neither guideline expressly suggests massage in patients with unexplained fall or drop attacks that are likely to be syncopal, it is our centre's practice to do so in such individuals given the discussion above±Five seconds massage, and more than three seconds asystole for test positivity in North American guidelines (idelines §The European guidelines suggest repeated CSM with intravenous atropine injection to distinguish predominant cardioinhibition from vasodepression in mixed sub-type CSS (type CSS  in order§Levels of evidence for pacing intervention are IIa in North American (American  and IIb American If a slavish adherence to a gold-standard evidence base is to be the sole guiding principle, pacing cannot be recommended for CSS. However, in the real world of patient care where clinical experience chimes with the weight of history, while further evidence is rigorously sought, it is not unreasonable to follow imperfect but sensible consensus guidance until an unambiguous verdict is reached .The author confirms being the sole contributor of this work and has approved it for publication.The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The American Thyroid Association (ATA) uses the GRADE or the American College of Physicians (ACP) system to develop recommendations. Recommendations based on low-quality evidence should spur for the conduction of clinical studies, if feasible. The extent to which recommendations by the ATA based on low-quality of evidence are being actively researched remains unknown.Clinical guidelines produced by the ATA using the GRADE or the ACP system to classify evidence were deemed eligible. Reviewers, in duplicate and independently, extracted therapeutic recommendations based on low-quality evidence, whereas recommendations with higher quality of evidence, aimed at diagnosis, or best practice statements were excluded. Eligible recommendations based on low-quality evidence were deconstructed to their components using the PICO format. We then searched on clinicaltrials.gov to identify ongoing research. Trials were deemed eligible if they addressed the PICO question with at least one of the intended outcomes.A total of 543 recommendations were retrieved, of which 305 (56%) were based on low-quality of evidence and only 90 were deemed eligible. Of these, we found that 33 (37%) recommendations were actively being researched in 53 clinical trials. Most of the trials were randomized and funded by non-profit organizations. Many clinical trials studied thyroid nodules and differentiated thyroid cancer , whereas few studied were aimed at anaplastic thyroid cancer .One out of three of gaps in evidence, identified as low quality during the development of ATA guidelines, are currently actively researched. This finding calls for the need to develop a better research infrastructure and funding to support thyroid research. When creating guideline recommendations, guideline panelists estimate the benefits and harms of relevant options and rate the confidence they have in those estimates. , 2 The GThese knowledge gaps present areas of opportunities for research that will likely have an impact on the quality of evidence, strength of future recommendations, and patient care. However, recent evidence suggests that these knowledge gaps often remain so due to lack of active research.  RecentlyTo that end, we performed a meta-epidemiological study on ATA guidelines aiming to describe recommendations supported by low-quality evidence and determine the extent to which these potential knowledge gaps are being actively researched.This study was conducted in alignment with an existing reporting guideline for meta-epidemiological studies . 17] A  A 17] A https://www.thyroid.org/professionals/ata-professional-guidelines/) and only guidelines using the GRADE or the ACP system to provide recommendations were selected. 16] InterLess than half of these recommendations  are being actively researched . CurrentMost of the trials were funded by non-profit organizations , had a parallel design , were randomized , and were actively recruiting  . Almost Less than 10% of the recommendations endorsed by the ATA guidelines stem from high-quality evidence while over 50% is based on low-quality evidence. Only one out of three knowledge gaps identified in management recommendations of common thyroid disorders are actively being researched.Our findings align with previous evidence elucidating the extent of which clinical guidelines are based on low-quality of evidence. , 23 MuraAn analytical assessment performed by Checketts et. al. on the American Society of Plastic Surgeon’s clinical guidelines for breast reconstruction found that 6 out of 10 identified research gaps were being evaluated by active research.  While, SWe utilized a protocol-driven method, that facilitates reproducibility however, our results have several limitations that may reduce the confidence in our estimates. First, there are several instances in which performing a clinical study may be unethical or not feasible and by excluding observational evidence, our results may underestimate the extent to which research is addressing current knowledge gaps. However, we purposefully excluded best-practice recommendations which reduce this possibility. Second, we excluded other trial registries but due to the scope of clinicaltrials.gov we are confident that excluding additional registries did not appreciably alter our estimates. Additionally, most guidelines were using the ACP system to qualify recommendations, and thus, lacked a “best practice” category. But as this system mimics the rationale and criteria of the GRADE system and experienced methodologists made the final evaluation, we consider this limitation to be insignificant. Despite these limitations, this approach has successfully been used previously and experts in methodology and clinical research planned and guided the conduct of this study,  thereby Based on our findings, we have found that most clinical recommendations, put forth by the ATA guidelines, are based on observational studies. This high prevalence may be due to several perks of thyroid research. For instance, conducting a multinational RCT requires for a multidisciplinary team that helps with the design, recruitment, analysis, and publication, and all these can only be afforded by proper funding. Yet, recent evidence has shown that thyroid research lacks the proper funding when compared to other areas with more prevalence and impact on morbidity and mortality.  This lacThere is an unquestionable valuableness of observational studies to yield causal results whenever conducted properly and adjusting for possible confounders. , 33 UsinOn the other hand, the conduction of an RCT’s are far from being exempt of bias.  Blindly De facto, the increasing prevalence of these conditions would be putative of causing billion dollars in healthcare costs in the following years. Reviewers' comments:Reviewer's Responses to QuestionsComments to the Author1. Is the manuscript technically sound, and do the data support the conclusions?The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: YesReviewer #2: Partly**********2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: YesReviewer #2: Yes**********3. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #1: YesReviewer #2: Yes**********4. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #1: YesReviewer #2: Yes**********5. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #1: This is an interesting and well written manuscript that highlights the reality of difficulties in reliance in evidence-based medicine and how much of what clinical guidelines  recommend is based on less than optimal evidence and the need to constantly review and revise guidelines as new evidence emerges.It would be very helpful  for the authors to highlight 3 specific areas or questions/recommendations in the field of thyroidology which would benefit from realistic RCT that would help advance the field. Many questions are not easily answered, others cannot be subjected to RCT for ethical reasons, etc.Reviewer #2: The authors have examined the American Thyroid Association guidelines (those which use the ACP system to classify evidence) to identify those recommendations with low-quality evidence and determine which of these are being researched. Of the 5 sets of guidelines, 56% were based on low-quality evidence and 90 were further examined. Overall, 37% were actively being researched in clinical trials, suggesting gaps in thyroid research and a call for consideration of increased funding.Comments:1. It would be helpful to have a supplemental table/index of those recommendations that the authors found to be 'best practices' or for 'diagnostic' purposes. I was not clear from their methodology what they meant by that term and how it reflected in the guidelines chosen.2. The authors discuss previous studies that have examined the pairing of strength of recommendation with quality of evidence; perhaps the authors could also elucidate which of the low-quality guidelines were associated with what strength of recommendation and if this correlated to the presence (or absence) of clinical trials;3. the authors encourages panelists on guidelines to fully endorse the GRADE guidance. I do not think this is appropriate in the context of this manuscript.**********what does this mean?). If published, this will include your full peer review and any attached files.6. PLOS authors have the option to publish the peer review history of their article  digital diagnostic tool,  12 May 2020Reviewer #1Comment #1:This is an interesting and well written manuscript that highlights the reality of difficulties in reliance in evidence-based medicine and how much of what clinical guidelines  recommend is based on less than optimal evidence and the need to constantly review and revise guidelines as new evidence emerges.It would be very helpful  for the authors to highlight 3 specific areas or questions/recommendations in the field of thyroidology which would benefit from realistic RCT that would help advance the field. Many questions are not easily answered, others cannot be subjected to RCT for ethical reasons, etcResponse:We appreciate the kind words and insightful observation of the reviewer. We recognize that the conduction of RCTs may be unfeasible due to funding, ethics, or other reasons, and that most of the times, the use of observational data is what is available as best evidence. Although it is impossible for this paper to provide a comprehensive review of current epidemiologic methods to draw causal statements from observational data, we have added a paragraph and several references that will guide the readers in case they are interested.The manuscript reads as follows:Based on our findings, we have found that most clinical recommendations, put forth by the ATA guidelines, are based on observational studies. This high prevalence may be due to several perks of thyroid research. For instance, conducting a multinational RCT requires for a multidisciplinary team that helps with the design, recruitment, analysis, and publication, and all these can only be afforded by proper funding. Yet, recent evidence has shown that thyroid research lacks the proper funding when compared to other areas with more prevalence and impact on morbidity and mortality.[31] This lack of funding impairs the proper conduction of RCT’s, and therefore, limits panelists to utilize the best available evidence to create clinical guidelines, despite the best stands for mostly being observational studies. But are observational studies always deemed to deploy low-quality of evidence? The answer is no. There is an unquestionable valuableness of observational studies to yield causal results whenever conducted properly and adjusting for possible confounders. Using observational data to make causal statements, however, requires the proper methods; causal questions requires causal method to answer them. For this, a rogue group of statisticians and epidemiologists defied the adage of “association is not causation” and stated that causality could be estimated from observational studies when proper methods are applied.[34-37] When this premises are violated, observational studies may yield an association between a treatment and an outcome even if this physiological plausibility fails to hold in real-world scenarios. For instance, one of the most famous cases that bespeak the latter concept was the use of hormone replacement therapy (HRT) to prevent CVD in postmenopausal women.[38] The Framingham Cohort Study showed that the incidence of myocardial infarction (MI) among premenopausal women was virtually inexistent. This led to the assumption that by treating postmenopausal women with HRT, the incidence of MI will decrease.[38] Yet, this surmise failed when tested in a RCT.[39] Along this example, there are several other studies that highlight the precautions of using observational studies to provide care when analyzed incorrectly. On the other hand, the conduction of an RCT’s are far from being exempt of bias.[41] Blindly inputting our faith towards everything that is labeled “randomized clinical trial” spurs for an inadequate application of evidence-based medicine to patient care.[42] A thorough and peruse appraisal of the evidence is necessary for clinicians to make confident clinical decisions and for panelists to develop trustworthy clinical recommendations. Additionally, some thyroid condition, especially rare diseases, with low prevalence, or high mortality, conducting a RCT would demand a tremendous effort, funding, recruitment, and some sort of fortunateness, and most likely, the resulting RCT will be underpowered, unrepresentative, and with misleading results. Observational studies may sometimes be the best and most feasible option to provide a causal inference and implement it to patient care.Reviewer #2Comment #1 It would be helpful to have a supplemental table/index of those recommendations that the authors found to be 'best practices' or for 'diagnostic' purposes. I was not clear from their methodology what they meant by that term and how it reflected in the guidelines chosen.Response:We have added a table as a supplementary file from a previous publication in which we provide examples from each case. Comment #2The authors discuss previous studies that have examined the pairing of strength of recommendation with quality of evidence; perhaps the authors could also elucidate which of the low-quality guidelines were associated with what strength of recommendation and if this correlated to the presence (or absence) of clinical trials.Responsehttps://www.jclinepi.com/article/S0895-4356(19)30836-4/pdf).We appreciate the reviewer comment. However, classification based on the underlying quality was made solely on the recommendations, not the guidelines. Most guidelines had a mixed between low-, moderate-, and high-quality of evidence. We, however, acknowledge that it is important to elucidate what proportion of low-, moderate-, and high-quality recommendations had each guideline, and this has been elucidated in a previous publication from our same research group :We thank the authors for their patience in the time it took for the review of the manuscript. Given the content of the paper, we wanted to ensure it got the most appropriate review.Kind RegardsReviewers' comments:Reviewer's Responses to QuestionsComments to the Author1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.Reviewer #2: All comments have been addressed**********2. Is the manuscript technically sound, and do the data support the conclusions?The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes**********3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: Yes**********4. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.The Reviewer #2: Yes**********5. Is the manuscript presented in an intelligible fashion and written in standard English?PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #2: Yes**********6. Review Comments to the AuthorPlease use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #2: (No Response)**********what does this mean?). If published, this will include your full peer review and any attached files.7. PLOS authors have the option to publish the peer review history of their article (If you choose “no”, your identity will remain anonymous but your review may still be made public.Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.Reviewer #2: No 29 May 2020PONE-D-19-35572R1 A Deeper Analysis in Thyroid Research: A Meta-Epidemiological Study of the American Thyroid Association Clinical Guidelines Dear Dr. Rodriguez-Gutierrez:I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. onepress@plos.org.If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact plosone@plos.org. For any other questions or concerns, please email Thank you for submitting your work to PLOS ONE.With kind regards,PLOS ONE Editorial Office Staffon behalf ofDr. Honey V. Reddi Academic EditorPLOS ONE