Many clinical predictive tools have been developed to diagnose traumatic brain injury among children and guide the use of computed tomography in the emergency department. It is not always feasible to compare tools due to the diversity of their development methodologies, clinical variables, target populations, and predictive performances. The objectives of this study are to grade and assess paediatric head injury predictive tools, using a new evidence-based approach, and to provide emergency clinicians with standardised objective information on predictive tools to support their search for and selection of effective tools.Paediatric head injury predictive tools were identified through a focused review of literature. Based on the critical appraisal of published evidence about predictive performance, usability, potential effect, and post-implementation impact, tools were evaluated using a new framework for grading and assessment of predictive tools (GRASP). A comprehensive analysis was conducted to explain why certain tools were more successful.Fourteen tools were identified and evaluated. The highest-grade tool is PECARN; the only tool evaluated in post-implementation impact studies. PECARN and CHALICE were evaluated for their potential effect on healthcare, while the remaining 12 tools were only evaluated for predictive performance. Three tools; CATCH, NEXUS II, and Palchak, were externally validated. Three tools; Haydel, Atabaki, and Buchanich, were only internally validated. The remaining six tools; Da Dalt, Greenes, Klemetti, Quayle, Dietrich, and Güzel did not show sufficient internal validity for use in clinical practice.The GRASP framework provides clinicians with a high-level, evidence-based, comprehensive, yet simple and feasible approach to grade, compare, and select effective predictive tools. Comparing the three main tools which were assigned the highest grades; PECARN, CHALICE and CATCH, to the remaining 11, we find that the quality of tools’ development studies, the experience and credibility of their authors, and the support by well-funded research programs were correlated with the tools’ evidence-based assigned grades, and were more influential, than the sole high predictive performance, on the wide acceptance and successful implementation of the tools. Tools’ simplicity and feasibility, in terms of resources needed, technical requirements, and training, are also crucial factors for their success.The online version of this article (10.1186/s12873-019-0249-y) contains supplementary material, which is available to authorized users. Clinical decision support (CDS) systems proved to enhance evidence-based clinical practice and improve healthcare cost-effectiveness –6. BasedTraumatic brain injury (TBI) is one of the most commonly presenting emergency conditions and is the leading cause of death and disability among trauma patients , 19. In When selecting a predictive tool, for implementation at their clinical practice or for recommendation in clinical practice guidelines, clinicians involved in the decision making are challenged with an overwhelming and ever-growing number of tools. Many of these tools have never been implemented or assessed for comparative effectiveness or post-implementation impact –31. CurrWhile there are many predictive tools that have been developed, to help clinicians rule out TBI among children at the ED, only a few were considered for use in clinical practice –24. TherThe primary objective of this study is to grade and assess paediatric head injury predictive tools using a new evidence-based framework for grading and assessment of predictive tools (The GRASP Framework). The secondary objective is to provide emergency clinicians with standardised objective information on clinical predictive tools to support their search for and selection of effective tools.Our study is composed of three parts. The first includes identifying paediatric head injury predictive tools, proposed in the literature, and their related published evidence. The second part includes grading these predictive tools using our new evidence-based approach and eligible published evidence. The third part includes conducting a comprehensive and objective analysis to answer the research question.We conducted a focused review of the literature on paediatric head injury predictive tools. The concepts used in the literature search included “paediatrics”, “head”, “injury”, “clinical prediction”, “tools”, “rules”, “models”, “development”, “validation”, “implementation”, and “evaluation”. The search was conducted for studies published in English language, with no specific time frame, using MEDLINE, EMBASE, CINAHL, and Google Scholar. The default time range of each database was used, including available publications since 1879, 1950, 1947, and 1937 respectively and up to January 2019. The search followed five steps. 1) Systematic reviews on paediatric head injury predictive tools were identified and retrieved. 2) Examining the systematic reviews; the primary studies, describing the development of the tools, were then identified and retrieved. 3) All secondary studies that cited the primary studies or that referred to the tools’ names or to any of their authors, anywhere in the text, were retrieved. 4) All tertiary studies that cited the secondary studies or that were used as references by the secondary studies were retrieved. 5) Secondary and tertiary studies were examined to exclude non-relevant studies or those not reporting the validation, implementation or evaluation of the tools. Additional file Each paediatric head injury predictive tool was evaluated using our newly developed framework for grading and assessment of predictive tools (abbreviated as GRASP) . EligiblAssigns A, B, or C based on the highest phase of evaluation. If a tool’s predictive performance, as reported in the literature, has been tested for validity, it is assigned phase C. If a tool’s usability and/or potential effect have been tested, it is assigned phase B. Finally, if a tool has been implemented in the clinical practice, and there is published evidence evaluating its post-implementation impact, it is assigned phase A.A numerical score, within each phase, is assigned based on the level of evidence associated with each tool. A tool is assigned grade C1 if it has been tested for external validity multiple times, grade C2 if it has been tested for external validity only once, and grade C3 if it has been tested only for internal validity. Grade C0 means that the tool did not show sufficient internal validity to be used in the clinical practice. Grade B1 is assigned to a predictive tool that has been evaluated, during the planning for implementation, for both of its potential effect, on clinical effectiveness, patient safety or healthcare efficiency, and for its usability. Grade B2 is assigned to a predictive tool that has been evaluated only for its potential effect, while if it has been studied only for its usability, it is assigned grade B3. Finally, if a predictive tool had been implemented then evaluated for its post-implementation impact, on clinical effectiveness, patient safety or healthcare efficiency, then it is assigned grade A1 if there is at least one experimental study of good quality evaluating its post-implementation impact, grade A2 if there are observational studies evaluating its impact, and grade A3 if the post-implementation impact has been evaluated only through subjective studies, such as expert panel reports.For each phase and level of evidence, a direction of evidence is assigned based on the collective conclusions reported in the studies. The evidence is considered positive if all studies about a predictive tool reported positive conclusions and negative if all studies reported negative or equivocal conclusions. The evidence is considered mixed if some studies reported positive and some reported either negative or equivocal conclusions. To decide an overall direction of evidence, a protocol is used to sort the mixed evidence into 1) Mixed evidence that supports an overall positive conclusion or 2) Mixed evidence that supports an overall negative conclusion. This protocol is based on two main criteria; 1) Degree of matching between the evaluation study conditions and the original tool specifications, and 2) Quality of the evaluation study. Studies evaluating predictive tools in closely matching conditions to the tool specifications and providing high quality evidence are considered first; taking into account their conclusions in deciding the overall direction of evidence.The final grade assigned to a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. The GRASP framework concept is shown in Fig. We identified five systematic reviews , 27, 28 PECARN rule was developed by Dr. Nathan Kuppermann in the US in 2009 and was tested successfully for internal validity [Using the GRASP framework and eligible evidence, we assigned grades to the 14 paediatric head injury predictive tools. The validity . The rulvalidity , 76, 77.validity , 69, 75.validity . Howevervalidity , 64. UsiCHALICE rule was developed by Dr. Joel Dunning in the United Kingdom in 2006 and was tested successfully for internal validity [The validity . The rulvalidity , 72, 77.validity , 80. Howvalidity , 69, 78.CATCH rule was developed by Dr. Martin Osmond in the US in 2010 and was tested successfully for internal validity [The validity . The rulvalidity , 72, 81.NEXUS II rule was developed by Dr. William Mower in the US in 2005, primarily for the diagnosis of adult head injury [The d injury , 89. Latd injury . The tood injury . Two stud injury , 85 and d injury . Using tPalchak rule was developed by Dr. Michael Palchak and Dr. Nathan Kuppermann in the US in 2003 and was tested successfully for internal validity [validity . A studyvalidity . One extvalidity . The rulHaydel rule was developed by Dr. Micelle Haydel in the US in 2003 [Atabaki rule was developed by Dr. Shireen Atabaki in the US in 2008 [Buchanich rule was developed by Dr. Jeanine Buchanich in the US in 2007 [ in 2003 , Atabaki in 2008 , and Buc in 2007 . The thrDa Dalt rule was developed by Dr. Liviana Da Dalt in Italy in 2006 [Greenes rule was developed by Dr. David Greenes in the US in 2001 [Klemetti rule was developed Dr. Sanna Klemetti in Finland in 2009 [ in 2006 , Greenes in 2001 , 45, and in 2009 . The stuDr. Kimberly Quayle in the US in 1997 [Dr. Ann Dietrich in the US in 1993 [Dr. Ahmet Güzel in Turkey in 2009 [ in 1997 , Dr. Ann in 1993 , and Dr. in 2009 , each trTherefore, they could not produce predictive rules with sufficient internal validity. Accordingly, the final grade assigned to these three rules is C0. A summary of the results of grading the 14 paediatric head injury predictive tools, using the GRASP framework, is presented in Table The PECARN rule was the only tool evaluated in post-implementation impact studies. The PECARN and the CHALICE rules were evaluated for potential effect on healthcare, while the remaining 12 tools were only evaluated for predictive performance. Three of these 12 tools were externally validated; CATCH, NEXUS II, and Palchak rules, three were only internally validated; Haydel, Atabaki, and Buchanich rules, and the remaining six tools; Da Dalt, Greenes, Klemetti, Quayle, Dietrich, and Güzel rules did not show sufficient internal validity to be used in clinical practice.Using statistical analysis, we explored possible correlations between different criteria of predictive tools and their evidence-based assigned grades. There is no correlation between the country of the tools’ development and their assigned grades. For example, the 10 tools developed in the US include some of the highest and some of the lowest grades, so the country of a tool’s development is not related to the grade of the tool. There is a weak correlation between the year of the tools’ development and their assigned grades. The tools developed more recently could be higher in grade. There is a strong correlation between the number of citations of the tools, in the literature, and their assigned grades. The tools with higher citations are expected to be higher in grade. There is a very strong correlation between the number of studies discussing the tools and their assigned grades. The tools discussed and reported in more studies are higher in grade.To provide clinicians with a few more objective measures to compare the tools, in addition to the citations and the published studies, we developed three derived values; the citation index, the publication index, and the literature index. The PECARN, the CHALICE and the CATCH rules were cited in the literature 885, 309, and 319 times respectively. To make these figures comparable, we calculated the citation index as the average annual citations for each tool, by dividing the total citations of each tool by its age in years. Similarly, the publication index is the average annual studies discussing each tool. We also calculated a literature index; by multiplying the total number of citations by the total number of studies, on each tool, divided by 1000, for simplification. This figure reflects the post-implementation impact of each tool in the literature. Like the citations and publications, the three indices of the tools are strongly correlated with their assigned grades.Looking at more detailed objective measures, reported in the development studies of the 14 paediatric head injury predictive tools; we find very interesting results. The predictive tools were developed using two main methodologies. Recursive partitioning was used to develop the PECARN, CHALICE, CATCH, NEXUS II, Palchak, Haydel, Atabaki, and Buchanich rules. Multivariate logistic regression analysis was used to develop Greenes, Da Dalt, Klemetti, Quayle, Dietrich and Güzel rules. In addition, many clinical variables were used in the development of the tools, such as altered mental status, amnesia, focal neurological signs, occurrence of seizure after injury, presence of skull fractures, loss of consciousness, history of headache and/or vomiting. The mix of clinical variables used, to build the tools’ predictive models and their outcome scores, were similar but not the same for any of the tools. Moreover, the tools development studies used different paediatric populations and sample sizes. Consequently, the predictive performances of the tools, such as their sensitivities and specificities, were variable. Most of the tools showed high sensitivities, with the majority ranging from 90 to 100%, while their specificities were very different; ranging from 15 to 87%.There is no correlation between the tools’ development methodologies and their predictive performances. However, most of the tools developed using recursive partitioning showed relatively higher sensitivities but not necessarily better specificities. In addition, there is no correlation between the tools’ development methodologies and their assigned grades. However, the six tools that used multivariate logistic regression analysis were all assigned grade C0; reporting no internal validity, while the other eight tools that used recursive partitioning showed higher variable grades. Furthermore, there is no correlation between the predictive performances of the tools and their assigned grades. For example, Da Dalt rule is assigned grade C0.However, it has the highest sensitivity of 100% and the highest specificity of 87% among all the tools. This could be explained by the fact that Da Dalt rule was not internally validated, which makes it unqualified for external validation or implementation. While the CHALICE rule, which is assigned grade B2, has a sensitivity of 98% and a specificity of 86%, we find that the PECARN rule, which is the highest tool, assigned grade A2, has a similar sensitivity of 97% but lower specificity of 59%.On the other hand, we find that there is a strong correlation between the size of the patient samples used in the development and internal validation studies of the tools and their assigned grades. The three main tools had the largest numbers of patients contributing to their development studies; 42,412 patients were enrolled and analysed to develop the PECARN rule, 22,772 to develop the CHALICE, and 3866 to develop the CATCH rule. The remaining 11 tools were developed using a relatively smaller number of patient samples, ranging from 3000 to only a hundred patients. In addition, there is a strong correlation between the number of researchers developing tools and their assigned grades. Two of the main three tools were developed by a large number of researchers; the PECARN rule was developed by 32 researchers and the CATCH rule was developed by 14 researchers. The remaining tools were developed by a relatively fewer number of researchers; ranging from 10 for the Palchak rule to only one researcher for the Buchanich rule.Moreover, there is a correlation between the impact factor of the journal that published the development studies of the tools and their assigned grades. The PECARN rule, for example, was published in the Lancet, which is a highly ranked journal with an impact factor of 53.3. Furthermore, the three main tools; the PECARN, the CHALICE and the CATCH rules, in addition to the NEXUS II rule, were all supported by dedicated and well-funded research networks, programs, and professional groups, such as the Paediatric Emergency Care Applied Research Network for the PECARN rule, the Children’s Head Injury Algorithm for the Prediction of Important Clinical Events study group for the CHALICE rule, the Paediatric Emergency Research Canada (PERC) Head Injury Study Group for the CATCH rule, and the National Emergency X-Radiography Utilization Study II for the NEXUS II rule. There is a correlation between being supported by dedicated research programs, as a tool, and having a higher assigned grade. A summary of tools’ information, development studies indices, predictive performance and quality indicators of the 14 paediatric head injury predictive tools is presented in Table Additional file This study presents a new evidence-based approach to grade and assess predictive tools. Based on the critical appraisal of the published evidence on predictive tools, the GRASP framework uses three dimensions to grade the tools: 1) phase of evaluation; before implementation, during planning for implementation and after implementation, 2) level of evidence; adding a numerical score within each phase, and 3) direction of evidence; positive, negative or mixed. The final grade is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. Among the 14 paediatric head injury predictive tools, the PECARN rule stands out clearly, since it is the only tool evaluated in post-implementation impact studies, which needs some explanation.The 14 predictive tools targeted variable paediatric age groups. Most of the tools focused on children less than 16 years of age. However, some tools extended their cover to less than 21 years, such as Atabaki, while others limited their population to children less than 2 or 3 years, such as Buchanich and Greenes. The tools used different development methodologies and their prediction models used different mix of clinical variables. Furthermore, the predictive performances of the tools, such as their sensitivities and specificities, were different. However, the predictive performances of the tools were not correlated with their assigned grades. This indicates that the technical specifications of the predictive tools did not, in the first place, influence their successful validation, acceptance, or implementation. The country and year of tools’ development were also non-significantly influential on their successful path from validation into implementation. On the other hand, the number of citations of the studies, describing the development of the tools, and the number of studies reporting them are clearly correlated with tools’ success. These two indicators are secondary to the main quality indicators of the tools’ development studies, such as the sample size of patients used in the development of the tools and the number of researchers developing these tools.In addition, the experiences of the researchers have an important role in leading better-quality studies. Three of the researchers who developed the PECARN rule have already contributed to older but less successful tools. Before leading the team to develop the PECARN rule in 2009, Dr. Kuppermann contributed to developing the Quayle rule in 1997 and the Palchak rule in 2003. Dr. Quayle and Dr. Atabaki each developed her own rule in 1997 and 2008, before joining the team in developing the PECARN rule in 2009. The affiliations of the researchers, to highly ranked institutes, and the support of the studies by dedicated and well-funded research networks, programs, and professional groups, added to the credibility of the tools among clinicians and organisations. As a result of the better quality and higher credibility, the PECARN rule development study was published in a top ranked journal with a high impact factor; the Lancet. In addition, the three main tools; the PECARN, the CHALICE and the CATCH rules were endorsed by professional organisations and recommended in clinical practice guidelines, such as the paediatric head trauma clinical guidelines developed by the Royal Australian and New Zealand College of Radiologists .Many studies compared paediatric head injury predictive tools. Among these, nine compared the three main tools; the PECARN, the CHALICE and the CATCH rules. Despite the fact that most of the studies reported PECARN as the highest quality tool, they reported that all three predictive tools had excellent sensitivities and performed well in assessing the outcome of clinically important TBI, suggesting that all were appropriate for use in assessing mild head injury in the ED , 91. HowSome predictive tools, in other clinical areas, gained their widespread acceptance and successful implementation by providing simplicity and feasibility. The Ottawa ankle and the Ottawa knee rules are good examples of simple paper based five items check lists, designed to exclude the need for an X-ray for possible bone fracture in adult patients at the ED , 96. TheAccordingly, selecting effective predictive tools remains a major challenge for most clinicians who usually lack the time and experience required to evaluate such tools; assessing their quality or grading their level of evidence, especially as their number and complexity have increased tremendously over the recent years. This is made worse by the complex nature of the evaluation process itself and the variability in the quality of published evidence. Furthermore, it is not always feasible to compare tools, even those designed for the same predictive tasks, due to the diversity of their development methodologies, clinical variables, target populations, conditioned applications, and predictive performances. Therefore, we chose not to look at the details of every single validation or implementation study. Alternatively, the GRASP framework provides users with a higher level and evidence-based approach to grade predictive tools through the critical appraisal of published evidence on their development and validation before implementation, usability and potential effect during planning for implementation, and post-implementation impact on clinical effectiveness, patient safety and healthcare efficiency. Based on the available evidence, the framework identifies tools that are more trusted by clinicians and researchers and consequently can be more successful. Using the GRASP framework might need some training for expert healthcare professionals and researchers, who are going to grade predictive tools and some awareness for end user clinicians who are going to use GRASP output to select predictive tools.The main limitations of this study include the possibility of missing some predictive tools which could have been developed by clinicians but not yet published, because the GRASP framework depends on grading predictive tools based on their published evidence. Similarly, some of the published predictive tools could have been implemented in clinical practice but no studies, reporting their implementation or evaluating their post-implementation impact, have been published yet. Furthermore, while this study is in press or soon after it is published, an evidence on some tools may become available and could have an influence on the assigned grade.Comparing the three main tools, which were assigned the highest GRASP grades PECARN, CHALICE and CATCH, to the remaining 11, we find that three main factors are highly crucial and indicate better tools. Firstly, the quality of the predictive tools, which is indicated by the development methodology of the tools, the patient sample size used for development, and the number of contributing authors. The quality is also reflected through the number of citations and number of studies discussing each tool. Secondly, the experience and credibility of the tools’ authors, reflected in their clinical specialty and affiliated organisations. Thirdly, the support by dedicated and well-funded research programs. These three factors were more significantly influential, than the sole high predictive performance, on the wide acceptance and successful implementation of the tools. In addition, tools’ simplicity and feasibility, in terms of resources needed, financial support, technical requirements, complexity and number of predictors, and training, are crucial factors of their success. It is important to select tools which best fit the intended tasks, the clinical conditions, the healthcare settings and the patient populations. Based on detailed specifications, a group of best predictive tools can be recommended for use in clinical practice. Through evidence-based grading of predictive tools, the GRASP framework confirmed the PECARN rule as the highest quality tool, compared to the other tools, which have variable levels of supporting evidence. The online availability of the GRASP framework will enable clinicians and clinical guideline developers to access detailed information, reported evidence and assigned grades of predictive tools. However, keeping such information up-to-date requires continuous updating of tools’ reports when new evidence becomes available.Additional file 1:Figure S2. Searching the literature for predictive tools and related published evidence. Figures S3 to S11. Statistical figures describing the fourteen paediatric head injury clinical predictive tools. Table S3. The GRASP Framework Detailed Report template. Tables S4 to S17. The GRASP Framework Detailed Report on each of the fourteen paediatric head injury clinical predictive tools. (PDF 958 kb)


Anthopleura japonica. AJLec was characterized using the hemagglutination assay, isothermal titration calorimetry (ITC), and glycoconjugate microarray analysis and we found that AJLec has a specificity for galactose monomers and β-linked terminal galactose residues in complex carbohydrates, but not for N-acetylgalactosamine , which is commonly recognized by galactose-binding lectins. The primary structure of AJLec did not show homology with known lectins, and a crystal structural analysis also revealed a unique homodimeric structure. The crystal structure of AJLec complexed with lactose was solved by measuring the sulfur single-wavelength anomalous diffraction (S-SAD) phasing with an in-house Cu Kα source method. This analysis revealed that the galactose residue in lactose was recognized via its O2, O3, and O4 hydroxyl groups and ring oxygen by calcium coordination and two hydrogen bonds with residues in the carbohydrate-binding site, which demonstrated strict specificity for the β-linked terminal galactose in this lectin.A novel galactose-specific lectin, AJLec (18.5 kDa), was isolated from the sea anemone,  Among them, C-type lectins and galectins are widely distributed and known to function in various carbohydrate recognition processes2. C-type lectins show a requirement for Ca2+ to bind to carbohydrates and have common carbohydrate recognition domains (C-type CRD) that show diverse carbohydrate-binding specificities. Many C-type lectins bear domains in addition to CRDs. Therefore, C-type lectins have 16 subfamilies, which are distinguished by their domain architectures3. In each of the subfamilies, the domains cooperate to perform a diverse range of functions, including cell-cell adhesion, immune response to pathogens, and apoptosis4. On the other hand, galectins show specific binding toβ-galactosides. Galectins have a unique motif that binds to carbohydrates in CRDs with a jelly-roll conformation. Many galectins have now been identified in animals, each of which is differentially expressed in various tissues. These galectins show a broad range of functions, including cellular adhesion, animal development, immune responses, and apoptosis5.Lectins are ubiquitously distributed in various tissues and body fluids of living organisms. In particular, lectins play a crucial role in the innate immune system of invertebrates through specific binding to polysaccharide-coated pathogenic bacteria. Lectins are classified into 48 families based on their common structures7. However, compared to the variety of marine invertebrate species that exist, knowledge about marine invertebrate lectins is very limited, till date.In innate immune systems of invertebrates, lectins are thought to play crucial roles as pattern recognition molecules through specific binding to polysaccharide-coated pathogenic bacteria. Some reports have suggested the presence of marine invertebrate lectins with characteristics that are much different from those of other organismsAnthopleura japonica. Lectins from sea anemone species have not been identified previously. Characterization of AJLec revealed strict specificity for galactose monomers and β-linked terminal galactoses in complex carbohydrates. Its amino acid sequence does not show any similarity with those of known lectins. Analysis of AJLec complexed with lactose revealed its binding mechanism and a structural basis for its specificity.In the present study, we purified a novel galactose-binding lectin named AJLec from the sea anemone A. japonica was used for an affinity chromatography with a lactose-Cellufine column. During the chromatography, a single peak appeared after elution with Tris-buffered saline (TBS) containing 100 mM lactose .Homogenized extraction form 2+ and Mn2+ ions and without any cations did not show any hemagglutinating activity, whereas that with Ca2+ showed significant activity with a minimum concentration of 0.1 mM  and 0.561 (lactose), these values may be underestimated because of the low association constants of the lectin8. In fact, the crystal structure of AJLec/galactose complex revealed that there is one carbohydrate-binding site in each polypeptide chain as described below.The purified AJLec showed significant hemagglutinating activity with rabbit, horse, bovine, chicken, or sheep erythrocytes  could also be accomplished by binding to β-linked terminal galactose residues in their carbohydrates.To investigate the binding specificity of AJLec for complex carbohydrates, a glycoconjugate microarray analysis was performed Fig. . Using aIn terms of binding to GalNAc-containing carbohydrates, no definite binding to any carbohydrates, including GalNAc and excluding terminal galactose residues, was shown . This indicates that AJLec does not bind to GalNAc residues in complex carbohydrates, which agrees with the results of the hemagglutination inhibition assay.Edman degradation of AJLec did not yield any N-terminal amino acids, suggesting that this lectin had a blocked amino terminus. Therefore, to obtain information on the internal sequence, the protein was cleaved with CNBr at methionine residues. Several peptides were obtained after separation by reverse-phase high-performance liquid chromatography (HPLC)  were then designed on the basis of this sequence and used for 3′-RACE to determine the 3′-terminal sequence of the cDNA. Additional amplification of the cDNA was performed by 5′-RACE using the primers R2 and R4, resulting in a complete AJLec cDNA sequence, as shown in Fig. mRNA purified from Nematostella vectensis (starlet sea anemone), whose whole genome has only been sequenced in sea anemone species, with 28–49% homology , whose whole genome sequence was recently decoded13, have homology with that of AJLec with 29–40%. Their expression has not been confirmed in these organisms. This suggests that proteins homologous to AJLec exist in sea anemone and coral species only. To elucidate its folding and the structural basis for its specificity, a structural analysis of the AJLec-lactose complex was carried out as described below.A Blast search for homologous proteins of AJLec revealed seven predicted proteins from p < 0.05; peptide score, 27; expected value, 0.0023).A peptide mass fingerprinting (PMF) analysis of the MS spectrum from a Pam-AJLec digest identified five major ions  as internal peptides and a minor ion  in the C-terminal peptide. The most prominent ion , which was not assigned by PMF, was 17 Da less than that of the sequence mass  of the theoretical N-terminal peptide (QR[Pam-C]GGWVK), suggesting that the N-terminal residue of mature AJLec is pyroglutamylated . The S-SAD with Cu Kα method utilizes the slight anomalous dispersion effect of sulfur that can cause difficulty in the phase determination of protein structures. To overcome this difficulty, we enforced high-redundancy data collection in a diffraction experiment and non-crystallographic symmetry (NCS) averaging in phase improvements. Electron density maps after the phase improvements yielded high enough quality images for tracing all residues of AJLec. The crystal structure of AJLec in complex with lactose was determined in the orthorhombic space group nit Fig. . Each of2+ ion, where its glucose residue was quite solvent exposed whilst its galactose residue was mainly buried in the binding site groove. In addition, one more Ca2+ ion was bound at a position, which was 10 Å apart from the sugar-binding site. The second binding site for the Ca2+ ion was composed of residues, Asn21, Asp142 Asn145, and Glu148, that was supposed to stabilize the overall structure  forming a disulfide-bonded dimer. Hemagglutination inhibition assay, ITC analysis, and glycoconjugate microarray revealed its strict specificity for β-linked terminal galactose, but not for GalNAc. This specificity of AJLec cause it to keep its distance from many known galactose-binding lectins showing the ability to bind to GalNAc or a low ability to bind to galactose. C-type lectins with QPD (Gln-Pro-Asp) motifs and galectin families are major examples of galactose-binding lectin families14. For example, CEL-I, a C-type lectin from Cucumaria echinata, shows affinity for GalNAc and galactose15. Galectins show a high binding constant for LacNAc 16 17, whereas that for galactose is relatively low 17. Peanut agglutinin PNA shows significant affinity for Galβ1-3GalNAc (49.0 × 103 Ka), which is much stronger than that for galactose (0.69 × 103 Ka)18. These reported galactose-binding lectins have a comparatively large specificity for more than one monosaccharide structure or strict specificity for particular structures of complex carbohydrates. To this date, AJLec was only found present in sea anemone and coral species throughout the course of evolution; therefore, the characteristics of AJLec should be useful for the specific environment of the sea anemone and coral, and its immune system.AJLec was purified from N. vectensis and S. pistillata, all the residues coordinating Ca2+ ion in the sugar-binding site are conserved. This indicates that the binding of Ca2+ ion is also crucial for function of the homologous proteins. On the other hand, residues interacting with galactose and coordinating with the second Ca2+ ion are not conserved partly in the homologous proteins. This indicates that the binding specificity in the homologous proteins would differ from that in AJLec, and coordination for the second Ca2+ ion would not be formed in the homologous proteins  and AJLec (this report) are first and second lectin structures, respectively, to be solved. In the determinations of both of these structures, we overcame the difficulties of the technique for high-redundancy data collection and NCS averaging for phase improvement. These practical accomplishments also showed that this technique does not require higher resolution data than that collected from crystals of moderate lectins (~1.5 Å resolution). This indicates that the S-SAD with Cu Kα method could be adapted for routine structural determinations of lectins and many other oligomeric proteins.The crystal structure of AJLec was solved by S-SAD phasing with an in-house Cu Kα source (S-SAD with Cu Kα). This method is extremely beneficial in preparing crystals and in terms of facilities. Namely, it does not require the preparation of any heavy atom derivatives or any facilities other than a typical in-house X-ray diffraction device. However, because of technical difficulties, with weak anomalous signals from S atoms at a wavelength of 1.54 Å, reported structures solved by the method have been rare until now. To overcome the problem of the weak anomalous signals, the S-SAD phasing was used to focus on low energy X-ray diffraction measurements in synchrotron beamlines20 showed some structural similarity with PA-IL (LecA) in Pseudomonas aeruginosa . PA-IL has similar characteristics as AJLec in its folds , function , and positions of sugar-binding sites in protomer structures  between AJLec and PA-IL. In addition, details of the binding specificities of the lectins differ, at least in part 22. Hence, the folding similarity did not evolve from a common ancestor, but was the result of convergent evolution of the lectins.The crystal structure of AJLec showed a disulfide-bonded dimer with a unique helical-rod like shape. The protomer structure indicated seven β-sheets forming a small jelly-roll type β-sandwich fold. A structure-based search using the DALI serverres Fig. 21. On th23. The N-terminal residues that interact for dimerization are distant from the sugar-binding site. Therefore, the modification in AJLec should not affect its sugar-binding activity, but could be related to the retention of the structure, stability, or secretion of the protein, although details of its function in AJLec are unclear.Sugar-binding sites were found to be located on both termini of the dimeric structure. The orientation of the sites showed a pattern common in dimeric lectins that indicated agglutination activity against polysaccharide-coated pathogenic bacteria. Pyroglutamic modification in N-terminal residues of AJLec was identified by mass analysis and confirmed by crystallography. This modification has been identified in a variety of proteins with known structural and functional roles2+ ion in the lactose-AJLec complex. Therefore, the Ca2+ ion could not form a coordinate bond with GalNAc, in which the 2′-OH residue was replaced by an N-acetyl residue. This would be the structural basis for the inability of AJLec to bind to GalNAc. This shows the importance of interactions between the 2′-OH group in galactose and residues in AJLec, whereas many other galactose-binding lectins do not show this interaction , which has homology with LecA, binds α-linked terminal galactose but not GalNAc26. Coordination pattern and interactions in Lectin A for α-galactoside are similar to that in LecA, while additional hydrogen bond with hydroxyl group at positions 2 in α-galactoside was confirmed in Lectin A. It would be a structural basis for an abolishment of binding in the case of GalNAc.In the comparison of AJLec and known calcium-binding lectins, C-type lectin and LecA (PA-IL), the known lectins were found to form coordinate bonds with CaLec Fig. . Most C-A. japonica. AJLec shows a unique specificity for galactose monomers and β-linked terminal-galactose residues, which differs from that of known galactose-binding lectins. The primary structure of AJLec was not homologous with that of any known lectins, indicating that this was a new type of lectin. The structural analysis of AJLec revealed a disulfide-bonded dimer structure with a unique helical-rod-like shape. The protomer structure of AJLec showed a jelly-roll type β-sandwich fold and a sugar-binding site containing a Ca2+ ion and bound-lactose molecule in the AJLec/lactose complex. The structure of the binding site and coordination pattern for the Ca2+ ion, which are dissimilar to those of any other type of lectin, conferred the specificity for β-linked terminal-galactose residues in carbohydrates. This characteristic of AJLec could lead to its use as a research tool and in medical procedures, because AJLec would have broad utility in profiling structures of complex carbohydrates28. In addition, the S-SAD with Cu Kα method used to solve the structure of AJLec might be routinely applied for structural determinations of lectins and many other oligomeric proteins.In summary, novel galactose-binding lectin AJLec was purified from A. japonica) from the Ariake Sea were purchased from a local dealer. Their bodies were stored at −20 °C.Sea anemones  column prepared as previously reported29. Frozen bodies of A. japonica (300 g) were suspended in 600 mL of TBS-Ca2+ , and disrupted in a blender, followed by centrifugation at 9,500 × g for 30 min. The supernatant was applied to the lactose-Cellufine column (10 mL) pre-equilibrated with TBS-Ca2+. After washing with this buffer, AJLec was eluted with 100 mM lactose in TBS. The protein was further purified by gel filtration on a HiLoad 26/60 Superdex 200 prep grade column  equilibrated with TBS and then eluted at a flow rate of 2.5 mL/min using an ÄKTAprime Plus apparatus . The eluate was dialyzed with TBS, concentrated to 1.0–10 mg/mL, and then used for subsequent experiments, except for crystallization. For crystallization of the lactose complex with AJLec, the purified protein solution, including 100 mM lactose and 1 mM CaCl2, was concentrated to 9 mg/mL.AJLec was purified from 2+ buffer. Then, 50 μL of 5% (v/v) erythrocyte suspension in the same buffer was added to each well. The plates were incubated for 1 h, and the formation of a sheet (an agglutination-positive result) or a dot (an agglutination-negative result) was monitored. The results are expressed as the minimal protein concentration required to produce visible agglutination. To examine the effect of divalent ions on AJLec hemagglutination activity, AJLec dialyzed with TBS buffer  were used for the hemagglutination assay. The lowest divalent ion concentration resulting in agglutination was determined by twofold serial dilution of solutions at a 25-mM initial concentration.Standard hemagglutination assays were performed using rabbit, horse, sheep, and bovine erythrocytes. The assays were performed in 96-well U-bottom plates with serial twofold dilutions of a 50-μL lectin solution with an equal volume of TBS-Ca200 microcalorimeter . The titration was carried out at 25 °C, with the ligand and AJLec dissolved in TBS-Ca2+. The measurements were performed in a cell with a volume of 200 μL with 20 injections of 2-μL ligands at 2-min intervals. Protein and ligand solutions were used at concentrations of 0.56 mM (10 mg/mL) and 5.6 mM, respectively. Control experiments were carried out to measure the ligand dilution-related temperature, which was subsequently subtracted from the ligand binding thermograms.Sugar specificity and thermodynamic parameters were measured by ITC using a MicroCal iTC30. Briefly, glycoproteins and glycoside-polyacrylamide conjugates were dissolved in a spotting solution  and spotted on a microarray-grade epoxy-coated glass slide (Schott AG) using a non-contact microarray-printing robot . AJLec was labeled with Cy3-NHS ester . The Cy3-labeled AJLec solution (10 μg/ml in TBS-Ca2+) was incubated with the glycoconjugate microarray at 20 °C overnight. After washing, images were immediately acquired using a Bio-REX Scan 200 evanescent field-activated fluorescence scanner .The sugar-binding specificity of AJLec for complex oligosaccharides was analyzed using a glycoconjugate microarray as described in a previous reportn-butylphosphine in 7 M guanidine-HCl, 10 mM EDTA, and 0.5 M Tris-HCl (pH 8.5) and pyridylethylated by treatment with 4-vinyl pyridine at 25 °C for 4 h in the dark. The resulting pyridylethylated AJLec was chemically cleaved at methionyl bonds with 1% CNBr in 70% (v/v) formic acid at 25 °C for 24 h by the method described by Gross31. The peptides generated by CNBr cleavage were separated by reverse-phase HPLC using HITACHI model L-6200 and L-4200 liquid chromatographs on a Wakosil-II 5C18 AR column . The column was equilibrated with solvent A , and the peptides were eluted at a flow rate of 1 mL/min using a linear gradient of 0–100% solvent B  at room temperature. Amino acid sequences of the separated peptides were determined using a PPSQ-21 protein sequencer (Shimadzu).The purified AJLec was reduced with tri-A. japonica were flash frozen in liquid nitrogen and ground to powder form. Total RNA was extracted with Isogen solution . Poly(A) RNA was collected using an Oligotex-dT30 mRNA purification kit , and cDNA was synthesized using a SMARTer cDNA Cloning Kit (Clontech). The desired DNA fragment was amplified by polymerase chain reaction (PCR) using two degenerate primers: DF3, 5′-TAYTAYYTIGAYCCITAYGTIATIAA-3′, and DR1, 5′-AARTAICCRAAIACRTCIGCRCA-3′. This DNA fragment was cloned into a pTAC-2 vector using E. coli JM109 cells (Clontech) and sequenced with an ABI PRISM 3130 Genetic Analyzer (Applied Biosystems). The amino acid sequence deduced from this DNA fragment agreed with the peptide sequences determined from the purified protein. Therefore, 3′- and 5′-rapid amplification of cDNA ends (3′-RACE and 5′-RACE) was preformed using primers F2, F3, R2, and R4, with F3 and R4 designed from this region using a SMARTer cDNA Cloning Kit and used for a nested PCR.The bodies of www.ncbi.nlm.nih.gov) were performed using the BLASTP algorithm in non-redundant databases of the National Center for Biotechnology Information (NCBI)32. Alignment of the sequences was performed using Clustal Omega software33.Online sequence homology searches  polyethylene glycol 3350) and subjected to vapor diffusion at 20 °C.Crystallization of AJLec was performed using the vapor diffusion method. Two to four microliters of the protein solution in TBS containing 100 mM lactose and 1 mM CaCl++ [RIGAKU]), whereas those of AJLec (high-resolution) were collected using a beamline BL-1A at the Photon Factory . Crystals of AJLec (S-SAD) and AJLec (high-resolution) were frozen at 120 K and 95 K, respectively, before data collection. The AJLec data set (S-SAD) was processed and scaled using MOSFLM34 and SCALA software35, respectively, and that from AJLec (high-resolution) was processed and scaled using HKL200036. The AJLec crystal data sets belonged to space group P212121, with two molecules per asymmetric unit.The AJLec data set (S-SAD) was collected in-house  data set was replaced by the AJLec (high-resolution) data set and anisotropic refinement was enforced. The lactose models were fit into the carbohydrate-binding sites according to the difference electron density map. The refinement statistics are shown in Table http://www.pymol.org). Secondary structure assignment was performed in DSSP40, and the interface surface area and assemblies of AJLec were calculated using PISA41.After failing in initial screens to generate heavy atom derivatives by soaking heavy atoms in AJLec crystals, we tried to determine the AJLec structure using the S-SAD with Cu Kα method. To overcome the difficulty of calculations with a slight anomalous dispersion effect from sulfur, we enforced high-redundancy data collection to obtain statistically processed data with sufficient accuracy for S-SAD measurements. The collected data set, AJLec (S-SAD), was used for phase calculation in PHENIX42. In brief, 20 µL of AJLec (approximately 1 mg/mL) was added to 20 µL of 2× SDS-sample loading buffer for Cys alkylation (125 mM Tris-HCl (pH 8.5), 20% glycerol, 4% SDS, 10% 2-mercaptoethanol, and 0.0025% bromophenol blue), and the protein solution was incubated at 95 °C for 10 min. Twenty microliters of 7 M acrylamide was added to the reduced protein solution and held for 1 h at room temperature. The resulting propionamidated (Pam)-AJLec solution (20 µL/well) was applied to SDS-PAGE43, and the protein band was stained with Coomassie Brilliant Blue. An in-gel enzymatic digestion was performed according to a method described by Shevchenko et al.44. The gel section was excised from the Pam-AJLec band using a gel picker , destained twice with 200 µL of 30% acetonitrile in 25 mM NH4HCO3, dehydrated with 100 µL of acetonitrile, and digested in-gel with lysyl endopeptidase  solution  at 37 °C for 16 h. Peptide extraction from the gel and sample loading onto a matrix-assisted laser desorption/ionization (MALDI) target plate was performed as described by Yamaguchi45. MS and MS/MS spectra were obtained using a matrix-assisted laser desorption/ionization (MALDI) quadruple-ion trap (QIT) time-of-flight (TOF) mass spectrometer  with 2, 5-dihydroxybenzoic acid  as the matrix in positive mode. MALDI-QIT-TOF mass spectra were externally calibrated using human angiotensin II  and human ACTH fragment 18–39  in a ProteoMass Peptide and Protein MALDI-MS Calibration Kit . Modification of the N-terminal peptide was analyzed by PMF and an MS/MS ion search using MASCOT ver. 2.3  with an original database  in our own MASCOT server. Search parameters used for PMF were the following: enzyme, Lys-C; fixed modifications, propionamide (C); variable modifications, oxidation (HW and M); mass values, monoisotopic; peptide mass tolerance, +/−0.3 Da; peptide charge state, 1+; and maximum number of missed cleavage sites, 2. Search parameters used for the MS/MS ions search were as follows: enzyme, Lys-C; fixed modifications, propionamide (C); variable modifications, oxidation (HW and M) and Gln → pyro-Glu (N-term Q); mass values, monoisotopic; peptide mass tolerance, +/−0.3 Da; fragment tolerance, +/−0.2 Da; max missed cleavages, 2. A protein score (PMF) > 51 and individual ion score (MS/MS ions search) >14 were considered significant (p < 0.05).Cysteine residues of AJLec were subjected to Cys-S-β-propionamidation using the method of Sechi and Chait with slight modificationswww.pdb.org (PDB code: 6A56). The nucleotide sequence of AJLec was deposited in DDBJ/EMBL/GenBank (accession number: LC339819). This work was performed under the approval of the Photon Factory Program Advisory Committee .The atomic coordinates and structural parameters of AJLec were deposited in the Protein Data Bank, Supporting Information


The role of mechanistic evidence tends to be under‐appreciated in current evidence‐based medicine (EBM), which focusses on clinical studies, tending to restrict attention to randomized controlled studies (RCTs) when they are available. The EBM+ programme seeks to redress this imbalance, by suggesting methods for evaluating mechanistic studies alongside clinical studies. Drug approval is a problematic case for the view that mechanistic evidence should be taken into account, because RCTs are almost always available. Nevertheless, we argue that mechanistic evidence is central to all the key tasks in the drug approval process: in drug discovery and development; assessing pharmaceutical quality; devising dosage regimens; assessing efficacy, harms, external validity, and cost‐effectiveness; evaluating adherence; and extending product licences. We recommend that, when preparing for meetings in which any aspect of drug approval is to be discussed, mechanistic evidence should be systematically analysed and presented to the committee members alongside analyses of clinical studies. We use the term “mechanistic evidence” when we refer to both types of evidence, for and from mechanisms.Note that the term “mechanistic evidence” is ambiguous. It can mean evidence of the existence or details of the operation of mechanisms, or it can mean taking previously established mechanisms as evidence to adduce other conclusions. informs the drug approval process in a wide variety of ways—a point that tends to be under‐appreciated and under‐described in the evidence‐based medicine (EBM) literature. Secondly, to argue that drug approval processes should explicitly include mechanistic evidence as part of the assessment of manufacturers' applications for licences and in postmarketing surveillance, so that it can be appropriately scrutinized and, if need be, challenged.This paper has two aims. First, to highlight the fact that mechanistic evidenceThis paper contributes to the general programme of making the role of mechanistic evidence in the health sciences more transparent and methodologically rigorous, so that decisions can be grounded in evidence. EBM has sought to improve health outcomes by making evaluation of clinical studies more rigorous. However, the results of clinical studies form only part of the evidence base. In particular, good quality evidence of mechanisms can be obtained from a wide variety of sources, not just clinical studies Table . The EBMDrug approval is a “hard case” for the thesis that one should explicitly scrutinize evidence for mechanisms. This is because it is common EBM practice to hold that when randomized studies are available, as they almost always are in the case of drug approval, they should be considered in preference to—or even to the exclusion of—other kinds of evidence. As we shall argue, this is not always appropriate: randomized studies may not measure (or may be underpowered to measure) outcomes of interest, either beneficial or, especially, harmful, and in any case other evidence, eg, from mechanisms, can support or undermine the results of such studies. Either way, mechanistic evidence should always be taken into consideration.This paper is structured as follows. First, in Section Before proceeding, it will be useful to define the key concepts to which we appeal, to avoid ambiguity.complex‐systems mechanism is a complex arrangement of entities and activities, organized in such a way as to be regularly or predictably responsible for the phenomenon to be explained.mechanistic process consists of a spatio‐temporal pathway along which certain features are propagated from the starting point to the end point.mechanism to refer to either a complex‐systems mechanism, or a mechanistic process, or some combination of the two. For example, the mechanism for pumping blood might be constituted by the complex‐systems mechanism of an artificial pacemaker for producing a timing signal, the complex‐systems mechanism of the heart itself, and the mechanistic process linking the two.A clinical study for the claim that A is a cause of B repeatedly measures the values of a set of measured variables that includes A and B. In an experimental study, the measurements are made after an experimental intervention. If no intervention is performed, the study is an observational study.A mechanistic study for the claim that A is a cause of B is a study that provides evidence of the details of the mechanism by which A is hypothesised to cause B. Note that a clinical study for the claim that A is a cause of C, where C is an intermediate variable on the mechanism from A to B, is also a mechanistic study for the claim that A is a cause of B, because it provides evidence of the details of the mechanism from A to B. A clinical study for the claim that A is a cause of B is not normally a mechanistic study for that claim, because, although it can provide indirect evidence that there exists some mechanism linking A and B, it does not normally provide evidence of the structure or features of that mechanism.A evidence for mechanisms includes evidence of either the existence of a mechanism or evidence of the details of a mechanism. While mechanistic studies provide evidence of the details of a mechanism, clinical studies can provide evidence of the existence of a mechanism. Thus, high quality evidence for mechanisms can be obtained by a wide variety of means, as shown in Table We emphasize here, as footnoted earlier, that effectiveness is a claim that a particular causal relationship holds in some target population of interest. A claim of efficacy is a claim that a particular causal relationship holds in some specific study population under particular controlled conditions. A claim of external validity (or applicability) is a claim that a particular causal relationship holds more widely than in a specific study population, controlled clinical setting, or experiment. Effectiveness is often established by establishing efficacy in a study population and then establishing external validity to a target population of patients.A claim of 2In the drug discovery process, mechanistic evidence is widely acknowledged to be crucial. Contemporary drug discovery and development are exceedingly “target driven” , starting with the characterization of a biological component that can serve as an intervention point to a disease mechanism, and proceeding to the design and synthesis or biological production of a compound able to interact with the target component. Once manufactured, a new compound needs to be evaluated for beneficial and adverse effects. This process is typically divided into phases Table .A improves some interim measure C, a so‐called biomarker, which is an indicator that the drug is likely to benefit the clinically relevant measure B). In phase III, the compound is tested in larger human trials in patients who have the disease. The latter part of phase II until the completion of phase III is focused on learning how best to use the drug in patients (determining appropriate dosing and learning how different patient characteristics influence dosing) and confirming that the drug is efficacious and sufficiently free of common harms in a sample of target patients.In the pre‐clinical phase (phase zero), a compound is tested in animals to determine an appropriate dose for human trials and to characterize any major organ toxicity. In phase I, the compound is tested in healthy human volunteers, unless the drug is likely to have adverse effects that obviate this . In phase II, the compound is tested in a small number of patients affected with the targeted disease. Phase I to midway through phase II is focused on learning what doses of the drug are tolerated and how the drug affects major organ systems, and confirming that the drug is likely to be effective for the proposed indication, called “proof of concept” .A successful phase III trial is a basis for applying for a marketing authorization . Sometimes, approval is conditional on conducting further studies after approval, typically to monitor unexpected adverse effects or reactions.Learning components of clinical drug development seek to answer key questions about the drug and its actions on the body. Examples of these questions include: What are the mechanisms by which the drug enters the body, distributes throughout the body, and is cleared from the body? What doses of the drug are pharmacologically active? With what biological systems does the drug interact and how does it affect these systems? The answers to these questions are determined by establishing the complex‐systems mechanisms and mechanistic processes at play. This knowledge then informs the design and interpretation of clinical studies, which seek to confirm that the drug is efficacious and sufficiently safe in the study population.Sheiner characterized clinical drug development as a series of “learn‐confirm” cycles.specific consumer given his or her characteristics? Is the drug effective in the kinds of patients who present to the clinic? Is the drug safe in patients with impaired renal or hepatic function, and what dose is appropriate? Are particular age groups at greater risk of adverse reactions? While partial answers to these questions will be provided by the studies conducted during the drug's development, it is important that both mechanistic evidence and evidence of clinical outcomes continue to evolve, especially given that clinical use of the drug will extend beyond the groups of patients represented in the original studies.What is understood about the actions of the drug gets progressively more sophisticated, and the evidence regarding its clinical benefits more compelling, as it successfully progresses through clinical development. This is due to the interplay between the emergence of evidence for mechanisms and confirming that the drug benefits patients in rigorously designed randomized trials. While Sheiner focused on clinical drug development, it is important to note that this interplay does not stop at drug approval. The questions that consumers, clinicians, and regulators have about medicines go beyond the evidence provided by even the most compelling phase III randomized trials. Will the drug benefit a There is an increasing focus on appropriate evaluation throughout a drug's life‐cycle. To do this well, to appropriately answer the questions of consumers, clinicians, and regulators, it is necessary to have reliable and relevant evidence for mechanisms and clinical outcomes. This goal is undermined by a tendency to define the best evidence for drug evaluation solely in terms of what provides the best evidence of clinical outcomes. In later sections, we provide specific examples of how mechanistic evidence informs the evaluation of drug efficacy and safety. Further examples are provided to demonstrate how drug evaluation benefits from making such evidence more explicit. Key benefits of this include better informing the interpretation and generalization of randomized trial results and identifying mechanistic assumptions that require further scrutiny.3Pharmaceutical tests are conducted to demonstrate that the drug, and its specific formulations, are sufficiently stable for clinical use. Key aspects that need to be determined are the rate at which the drug product loses potency and the identification and properties of any degradation products. The shelf life/expiry date of a drug product is determined by considering the rate at which potency is lost and the presence and toxicity of any degradation products. In the absence of toxic degradation products, the expiry date of a drug product is typically set such that the drug will retain greater than 90% of its labelled potency for the duration of its shelf life under recommended storage conditions.Knowledge and evidence of mechanisms play a central role in ensuring and assessing the stability of drug formulations. Knowledge of the chemical characteristics of the drug inform the way that the drug will be formulated and stored. Key stability tests are undertaken on the medicinal product in the selected storage container when stored as recommended. For example, glyceryl trinitrate (nitroglycerin) is highly volatile, and tablets tend to lose potency over time. Glyceryl trinitrate tablets need to be stored in glass containers with a foil lined cap, because loss of potency will be exacerbated if the tablets come in contact with plastic or other permeable packaging material; cotton wool, often included in drug containers, must not be packaged with glyceryl trinitrate tablets.The general approach to stability testing has developed in response to developments in the understanding of mechanisms of degradation. Stability tests assess degradation of the medicinal product over time under variations in temperature, humidity, and light. Two key stability tests include a long‐term test under recommended storage conditions  and an accelerated test . The long‐term test provides information on degradation under recommended storage conditions and the accelerated test provides information on degradation when the product is stored outside recommended conditions. It is important to understand likely degradation under deviations from recommended conditions, because some deviation is likely during the shelf life of the product—during storage by the manufacturers/wholesalers, distribution to pharmacies and hospitals, and the far less well controlled storage conditions provided by consumers.Finally, specific stability tests conducted for a drug product depend on the particular physiochemical properties of the drug and what is known or learned about the specific mechanisms of degradation to which the drug is susceptible. Stress tests early in drug development are important in providing evidence of degradation pathways and providing an opportunity to test degradation products for clinical and adverse effects. This information guides decisions regarding appropriate storage and appropriate stability testing. This is especially important for biologic medicines, because of the concern that degradation products may cause an immune response. Another example of mechanisms of degradation informing appropriate stability testing are water‐based drug products packaged in semipermeable containers. In addition to the routine stability tests outlined previously, these products also require tests to demonstrate that water loss under conditions of low relative humidity do not occur.4Currently, the mechanistic evidence that is always systematically evaluated in the drug approval process consists of studies of pharmacokinetics (PK) and pharmacodynamics (PD). Below, we briefly describe the roles of these, together with the closely related field of pharmacogenetics. Pharmacokinetics is the study of how a drug enters, distributes within, and clears the body. Pharmacodynamics is the study of how varying concentrations of the drug in the body produce therapeutic and adverse effects. Pharmacogenetics is the study of the genetic influences on drug pharmacokinetics and pharmacodynamics. Together, these sciences provide insights into the complex‐systems mechanism(s) that influence the concentration of the drug in the body and the relationship between the concentration of the drug and the drug's effects. Knowledge of a drug's pharmacokinetics, pharmacodynamics, and pharmacogenetics is rarely complete, but rather accumulates throughout drug development and subsequent clinical use. Pharmacokinetic and pharmacodynamic considerations are vital for making a number of key decisions during drug development. Specifically: What range of doses should be used at first in human studies? Which dose or doses should be selected for testing in phase III pivotal studies? What are the potential adverse effects of the drug and adverse reactions to it? For these reasons, PK and PD are among the types of mechanistic evidence required for a proper evaluation and interpretation of trial evidence. At these points, evidence from experimental animal models is used to estimate the likely response in humans. How reliable this extrapolation is depends on the relevant similarities between humans and the animal models used, comparable to evaluating external validity of trials  modelling is an example of these approaches. PBPK explicitly starts with a structural model based on the complex systems mechanisms involved in absorbing, distributing, and eliminating drugs.Understanding the mechanisms that explain a drug's pharmacokinetics remains important for determining appropriate doses for individual patients following drug approval. For example, the approved dosage recommendations for the anticoagulant enoxaparin in the treatment of deep vein thrombosis are typically stated relative to body weight. This is because elimination of enoxaparin is influenced by renal function and metabolism, which tend to vary in predictable ways with body weight. The challenge, however, is that lean body weight is a better predictor of the clearance of enoxaparin than total body weight. The distinction is unimportant in the leaner patients that are often enrolled in clinical trials, but critical in the broader range of patients treated in routine care.6For approval, a drug must be shown to be efficacious in patients with the targeted disease or condition. Phase III  trials are meant to demonstrate this sufficiently well to merit licensing. Demonstrating efficacy requires showing that the treatment is correlated with improvement in the condition, and that any observed difference between the treatment and control groups is attributable to the treatment. The latter requires sufficient evidence for ruling out explanations of the correlation in terms of chance, bias, or confounding, so that the only remaining explanation is that there is a mechanism linking the intervention and the outcome that shows how the former is at least partly responsible for the latter. An ideally conducted trial would provide this evidence directly: if a sufficiently large correlation were observed in a perfectly randomized, perfectly representative, sufficiently large trial, that would provide very strong evidence that the correlation is causal, ie, that there is some mechanism of action that gives rise to the correlation. In practice, however, studies tend to be imperfect in various respects and so less conclusive. In such cases, it can be useful to consider the evidence in favour of the hypothesised mechanism of action. A well‐established mechanism of action can support the efficacy claim, while a hypothesised mechanism that has little evidence or contrary evidence  can undermine the efficacy claim.At present, mechanistic considerations tend to be treated rather unsystematically at drug approval meetings. Often “the evidence” is taken to consist of reports of phase III trials, which are selected and analysed in detail in advance of the approval meeting, and subjected to further scrutiny at the meeting. On the other hand, discussion of mechanisms occurs principally at the meeting itself, mediated through the opinions of the experts and without its role or relevance being clear to all participants. The fact that evidence for mechanisms is part of the evidence base and can be crucial to evaluating efficacy is not widely recognized. However, such evidence can be analysed as systematically as evidence from phase III trials.http://www.ema.europa.eu/docs/en_GB/document_library/Summary_of_opinion_-_Initial_authorisation/human/003916/WC500216075.pdf, a treatment for osteoporosis with active ingredient teriparatide, was http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/003916/human_med_002060.jsp&mid=WC0b01ac058001d124 by the European Medicines Agency (EMA) without a major new study, on the grounds of biosimilarity with http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/medicines/000425/human_med_000798.jsp&mid=WC0b01ac058001d124, a different formulation of teriparatide.One obvious example of the crucial role for evidence for mechanisms in judgements of efficacy occurs when determining biosimilarity. A biosimilar is a biological medicine that is very similar to another biological medicine that has already been approved for use. Often the burden of proof in phase III trials is much lower for biosimilar drugs than for other drugs. Instead, those assessing biosimilarity rely more on evidence of similarity of mechanism of action, particularly evidence of similarity of structure and function.in vivo biological similarity of different formulations of the same compound, typically small molecules. Again, the burden of proof in the terms of clinical trials is lowered, subject to appropriate mechanistic evidence: principally, evidence of pharmaceutical and pharmacokinetic properties of the two formulations.While biosimilarity refers to similarity of complex biological molecules, bioequivalence refers to in vitro system to purification, not just the drug.Standards of proof are different to that for biosimilarity, because small molecules are accurately reproducible, and two formulations of the same medication will contain exactly the same active molecule, even though inactive excipients may differ. On the other hand, complex molecules are sensitive to small differences in product engineering, which may affect the pharmacokinetics and pharmacodynamics of the compound. Biological medicines are typically more complex than 100% synthetic compounds, and so extra scrutiny is required in establishing biosimilarity. Biosimilarity can depend on the whole manufacturing process starting from the choice and engineering of the Evidence for mechanisms also informs judgements of efficacy when evaluating the design of clinical studies and the appropriateness of the inferences drawn from their results. Determining whether a study design is of high quality and is based on sound science requires evidence for mechanisms, notably when assessing the diagnostic categories used in a study, whether the length of the trial was appropriate to demonstrate efficacy, and whether all plausible confounders were controlled for.As an example of the use of evidence for mechanisms to assess the inferences drawn from clinical studies, consider the case of quetiapine. A well‐accepted standard for assessing the efficacy of an antidepressant is to conduct a randomized trial and compare the effects of the antidepressant against placebo on the sum‐score of a depression rating scale. Commonly used rating scales include the Hamilton Depression Rating Scale and the Montgomery‐Åsberg Depression Rating Scale. Each of these scales consists of items assessing common symptoms of depression, including feelings of sadness and effects on sleep and appetite. There is increasing recognition of the limitations of focussing on sum‐scores in assessing depression and response to treatment.Quetiapine is an antipsychotic drug that was originally developed for the management of schizophrenia but has also been licensed for use in major depressive episodes in bipolar disorder . The application for approval was supported by several trials of quetiapine in patients with major depressive disorder. These trials met the methodological requirements for assessing antidepressant efficacy. However, mechanistic considerations raise a number of questions regarding the validity of the accepted standard for assessing antidepressant efficacy. Most relevant is the effect of quetiapine on sleep. Two trials report the effects of quetiapine on individual Montgomery‐Åsberg Depression Rating Scale items.promotes atherosclerosis through the formation of fatty plaques in the arterial wall, and HDL cholesterol prevents atherosclerosis by facilitating the removal of cholesterol from the arterial wall. The epidemiological evidence is consistent with this understanding: people with higher concentrations of LDL cholesterol tend to have higher rates of cardiovascular disease and people with higher concentrations of HDL cholesterol tend to have lower rates. Furthermore, there is very strong evidence from clinical trials that drugs that lower LDL cholesterol, such as statins, reduce the risk of cardiovascular disease. The same effect, however, has not been demonstrated for drugs that raise HDL cholesterol without affecting LDL.The development of dalcetrapib provides a striking case of the interplay between evidence for mechanisms and evidence from clinical trials when assessing efficacy. Dalcetrapib causes an increase in high density lipoprotein (HDL) cholesterol concentrations. The properties of HDL cholesterol and low density lipoprotein (LDL) cholesterol are complex, but relatively well understood.After showing promising results in early clinical development, dalcetrapib failed to demonstrate efficacy in a large phase III study.712), and enzymes . In some cases, the therapeutic target is not known but must exist; for example, the therapeutic target for lithium is not known, although the enzyme inositol‐1‐phosphatase, which it inhibits, is a strong candidate.All efficacious drugs have targets by which they produce benefit. Targets are usually tissue proteins, such as membrane‐bound or intracellular receptors, ion transporters or channels, and enzymes. A few drugs are used as replacements for deficient or absent endogenous substances, such as hormones , minerals , vitamins , in which the reactions occur at doses or concentrations lower than those associated with benefit;collateral reactions (orange), in which the reactions occur at doses or concentrations in the same range as those associated with benefit;toxic reactions (red), in which the reactions occur at doses or concentrations higher than those associated with benefit, either through the same mechanism (solid line) or some other mechanism (dotted line).In Figure The solid lines show on‐target effects, the dotted lines off‐target effects.Apart from adverse reactions that occur through exaggeration of the target effect . The mechanism by which this prolongation occurs is through inhibition of the cardiac potassium channels known as human ether‐a‐go‐go‐related gene channels. Knowing this, it is now routine practice for all drugs to be screened for inhibitory effects on human ether‐a‐go‐go‐related gene channels using, for example, sheep cardiac Purkinje fibres in vitro. If the outcome is positive, the drug is not further developed.Pharmacogenetic mechanisms can contribute to the risks of adverse reactions. For example, there are interindividual variations in the activities of enzymes of the CYP family, depending on polymorphisms in the relevant genes. Knowing in advance the pharmacokinetic mechanisms whereby a drug is metabolized allows predictions about adverse drug interactions and risks of adverse reactions in particular populations. Similarly, pharmacodynamic polymorphisms can help predict the risks of adverse reactions to some medicines.Dimethyl fumarate (Skilarence), a treatment for psoriasis, was considered by the Commission on Human Medicines (CHM) of the UK Medicines and Healthcare products Regulatory Agency (MHRA) on 24 March 2016. It concluded that the mechanisms of action (anti‐inflammatory and immunomodulating effects) were well established, but that further mechanistic studies were needed to investigate the risk of harms, particularly because carcinogenicity had been observed in animals. In addition, mechanistic evidence suggest that dimethyl fumarate alters lymphocyte function and (by extrapolation from treatments for multiple sclerosis) may increase susceptibility to progressive multifocal leukoencephalopathy, a fatal viral inflammatory disease of the brain caused by the JC virus , which if present in the body can be reactivated when immune function is impaired, for example by medication. Skilarence was eventually approved by the Committee for Medicinal Products for Human Use (CHMP) of the EMA in April 2017.8For a drug to be approved, the regulator must assess the external validity of the trial evidence submitted in support of the benefit to harm balance of the candidate drug: can the results obtained in a trial population be applied to the population for which the drug is to be licensed? In the ideal case, the trial and target populations are the same or similar in most respects, so that no differences between the populations can be expected. This is very rarely the case, but if in doubt, the regulators may require more clinical trials with more representative samples to be conducted before approval.The problem of external validity is compounded when the target population is, for example, small children, pregnant women, or patients with comorbidities. These groups are not typically included in clinical trials for ethical reasons, or, in the case of comorbidities, because of potential confounding by their inclusion. In such cases, the same features that prevent the use of clinical trials in specific patient populations consequently make those patients highly dissimilar to the average trial subjects and therefore vitiate external validity. One must then use means other than clinical trials to assess whether those dissimilar features of the target population will make the patients react differently to the drug, possibly undermining assessment of the benefit to harm balance.Mechanistic evidence features crucially in evaluating external validity of trials. Differences in the mechanisms underlying the effects of a drug, or in features that could interfere with the mechanisms, can undermine the application of the trial results to the target population.https://www.sps.nhs.uk/medicines/abaloparatide/ by the CHMP as of July 2017.Abaloparatide is a treatment for osteoporosis in post‐menopausal women who have a high risk of fractures. It was considered by the CHM on 24 March 2016. The argument centred on extrapolation: a clinical trial showed that abaloparatide reduced the risk of vertebral fractures;Sofosbuvir/velpatasvir is a combination therapy for hepatitis C. The CHM considered this treatment on 24 March 2016 but raised a safety concern on the grounds of extrapolation: velpatasvir has been found to cause serious teratogenicity across three species , and this robustness across species was thought to provide significant evidence of a possible teratogenic effect in humans. Robustness of effect is important evidence of similarity of mechanism.9If the benefit to harm balance of a medication is acceptable, and the manufacturer receives a licence to market it, there remains the question of whether a health care system can afford to use it to treat members of its population, given that health care budgets are limited. One way of deciding this is to calculate the cost‐effectiveness of the medication, ie, whether the effect it offers gives good value for money. The usual method for doing this is to calculate the overall cost of using the medication and dividing it by a measure of the quality of life that is gained by using it. The quality of life is assessed by a measurement called the quality adjusted life year or QALY. A QALY of 1 implies perfect health and a QALY of 0 implies no health at all . QALYs are typically measured using instruments that elicit patients' answers to questions about their health. For example, one such instrument, the EQ5D, asks how problematic the individual finds mobility, self‐care, usual activities, pain/discomfort, and anxiety/depression. The difference between the QALYs before and after treatment, the QALY gain, is divided into the cost, giving an incremental cost‐effectiveness ratio (ICER). In the UK, if an intervention has an ICER of £20 000 to £30 000 per QALY gained, it is considered to be cost‐effective and can be recommended for funding by the health care system.Mechanisms are often not discussed by committees charged with determining the cost‐effectiveness of therapeutic interventions, but understanding mechanisms can influence decisions in various ways.In constructing pharmacoeconomic models that relate clinical outcomes to costs, it may be helpful to include mechanistic considerations. For example, in a multiple comparison of different types of antihypertensive drugs a decision will have to be made about whether to compare drugs with different mechanisms of action  and whether, within a pharmacological class, to include compounds with variable actions .If an intervention is claimed to be efficacious but the proposed mechanism of action is not biologically plausible, or is biologically implausible, or if there is no well attested mechanism, the claim of efficacy may be vitiated and the size of the QALY gain put in doubt.In some cases, conflicting analyses can be informed by an appeal to mechanisms. For example, in an indirect comparison of two medications that both increased platelet counts in children with idiopathic thrombocytopenia, an analysis by the manufacturer of one of the medications suggested that there was no significant difference between the two compounds, while an independent analysis suggested otherwise.10The reasons people seek treatments, the reasons they adhere to the treatments offered, and the interaction between help seeking and subsequent adherence to treatment have been extensively investigated over many decades.The human actions involved take place in a social milieu that varies widely in terms of infrastructures , the competencies of staff and patients, and the meanings that people attribute to what is going on.The method traditionally used to deal with poor adherence is intention to treat analysis. This is the statistical device of including all people who entered a trial in the final statistical analysis, even if they dropped out, stopped talking the medication, or took it variably. This gives a proxy for how well the drug will work in the total population on average. It is outcome focussed. This is of course better than simply assuming that what works under controlled laboratory or clinical settings will work in the field. But it skates past the behavioural mechanisms that are involved and remains locked into looking at associations, correlations, and averages in total populations, not at mechanisms.It follows from this that in the drug approval process there is a set of mechanisms that involve human behaviour. Just as the sciences of pharmacokinetics and pharmacodynamics constitute an intrinsic part of the process of approval, so too should the sciences of psychology and sociology, along with epidemiological and statistical methods and intelligence about industrial manufacture, quality control, and packaging. Formally, with the exception of the behavioural sciences, these all constitute parts of the decision‐making process for drug approval. What needs to be guarded against is a default to common sense assumptions or various forms of heuristic thinking about behavioural matters because that inevitably leads to bias and mistakes.Despite many efforts to understand the mechanisms underlying poor adherence to medications and to improve it, adherence continues to pose a major problem in the interpretation of data in pivotal trials during drug development. Variable adherence can have major consequences in clinical trials, resulting in confusion in their interpretation and reduced accuracy of estimates of benefit and harms. This can adversely affect drug development, especially of treatments that have a narrow therapeutic window. The consequences can include failure to confirm efficacy, underestimation of efficacy or of the risks of harms, emergence of drug‐resistant organisms in trials of anti‐infective agents, and impaired development of breakthrough drugs and drugs for rare diseases.11Extending a product licence (marketing authorization) to new indications and/or new populations requires regulatory approval. The following examples illustrate the difficulties of assessing licence extension and the role that evidence for mechanisms and inferences based on presumed mechanisms play in assessing the desirability of extension.Extending the indication of a treatment already on the market typically requires new clinical trial evidence of the benefit to harm balance of the treatment for the new indication. The importance of this is illustrated in cases in which treatments have been adopted for new populations or indications that have subsequently been shown to be ineffective or insufficiently safe. The use of antidepressants in adolescents follows this outline. Before 2005, antidepressants were used extensively in adolescents with depression. This practice rested, at least in part, on the assumption that the benefit to harm balance in adolescents would be similar to that seen in clinical trials in adults with depression. While this remains controversial, systematic reviews and evidence made available through legal cases have brought into question the efficacy and safety of several antidepressants in adolescents, as well as the reporting of several sponsor‐initiated studies.The extent to which antidepressants were prescribed for adolescents in the belief that the benefit to harm balance was the same as in adults represents an example of failed extrapolation. This could have been addressed by waiting for compelling evidence from clinical trials before use. However, this response is not entirely satisfactory, given the challenges of treating depression and the continuing uncertainty regarding harms, specifically the risk of suicidal ideation. Questions remain about appropriate measures of suicide risk, when suicide and attempted suicide are attributable to treatment, and there is evidence of poor data practices, publication bias, and, as has been suggested in some cases, data manipulation in relation to suicide risk.in vitro evidence that provides insight into the disease mechanism. Because the functional defect—loss of regulation of ion and water transport—is known, and the mechanisms responsible for it are fairly well characterized, in vitro assays demonstrating that cells regain function in the presence of a drug are expected to provide a good biomarker of clinical success. Laboratory evidence of this effect in different CFTR mutant cells, together with trial evidence for previously approved indications, allowed the FDA to conclude that the drug will work in several cystic fibrosis genotypes not tested in clinical trials.In other cases, mechanistic evidence may be available. The recent extension of the licence for ivacaftor in cystic fibrosis provides a striking example. In May 2017, the FDA expanded the licence for ivacaftor to several new genetic variants of cystic fibrosis. Cystic fibrosis results from a defective transmembrane conductance regulator (CFTR) gene, which encodes a protein that facilitates the transport of salt and water in and out of cells. Specific mutations that either block the CFTR gene from being transcribed or affect the protein product in particular ways will cause loss of functionality, generating the clinical manifestations of cystic fibrosis. The nature of the symptoms varies, depending on which CFTR allele is affected. Different drugs that work to compensate the defect therefore vary in their effectiveness against different cystic fibrosis genotypes. Many of the mutations are so rare that conducting clinical trials to test the effectiveness of a drug against specific genotypes would be impractical, if not impossible. Instead, in its decision, the FDA relied extensively on 12Evidence‐based medicine seeks to make evidence explicit and to develop explicit methods for evaluating it. In practice, present‐day EBM focuses almost exclusively on clinical studies—it treats mechanistic evidence that arises from other sources as irrelevant or peripheral. But mechanistic evidence is neither of those things: we have argued that evidence for mechanisms, ie, evidence that mechanisms exist and how they operate, is central to drug approval, because it informs the drug approval process in a wide variety of ways. We believe that the drug approval process would benefit from explicitly including mechanistic evidence as part of the assessment of manufacturers' applications for licences and in postmarketing surveillance, so that it can be appropriately scrutinized and, if need be, challenged. The EBM+ programme explores ways of making evidence of mechanisms explicit and suggests ways of using such evidence to inform causal evaluation (see, for example, ref While Parkkinen et al