As the most widely-used single cell whole genome amplification (WGA) approach, multiple displacement amplification (MDA) has a superior performance, due to the high-fidelity and processivity of phi29 DNA polymerase. However, chimeric reads, generated in MDA, cause severe disruption in many single-cell studies. Herein, we constructed ChimeraMiner, an improved chimeric read detection pipeline for analyzing the sequencing data of MDA and classified the chimeric sequences. Two datasets (MDA1 and MDA2) were used for evaluating and comparing the efficiency of ChimeraMiner and previous pipeline. Under the same hardware condition, ChimeraMiner spent only 43.4% (43.8% for MDA1 and 43.0% for MDA2) processing time. Respectively, 24.4 million (6.31%) read pairs out of 773 million reads, and 17.5 million (6.62%) read pairs out of 528 million reads were accurately classified as chimeras by ChimeraMiner. In addition to finding 83.60%  chimeras, which were detected by previous pipelines, ChimeraMiner screened 6,736,168 novel chimeras, most of which were missed by the previous pipeline. Applying in single-cell datasets, all three types of chimera were discovered in each dataset, which introduced plenty of false positives in structural variation (SV) detection. The identification and filtration of chimeras by ChimeraMiner removed most of the false positive SVs (83.8%). ChimeraMiner revealed improved efficiency in discovering chimeric reads, and is promising to be widely used in single-cell sequencing. Among the single cell whole genome amplification (WGA) approaches, multiple displacement amplification (MDA)  is consiThe phi29DNApol-mediated chimeras were first discovered in 2006 , and werIn previous works ,9, authoThe bioinformatics pipeline above is complicated, the alignment and the circulation of extension take lots of time. In addition, the split-reads generated by the SOAP2 were not used for chimera analysis in the previous pipeline, which might miss plenty of single-end chimeras. Hence, we designed an improved bioinformatics pipeline to recognize chimeras from alignment results, and realized accurate classification and quantity statistics of chimeras, called ChimeraMiner. ChimeraMiner was built on the Burrows-Wheeler Aligner (BWA) ,30,31, aWe filtered the reads with “N” bases from all datasets, and the cleaned reads were used for alignment. Statistics of data and alignment results are provided in We used both of the methods on two MDA samples to discover and classify the chimeras, SRX247249 and SRX252522  . The single-end chimera was further divided into direct chimera and inverted chimera, based on the orientation of the two subsections of the chimera. Normally, single-end chimeras were undoubtedly abandoned in almost studies. In fact, these chimeras could be converted into useful data by splitting from the chimeric sites after detection. Therefore, our study improves the utilization efficiency of the sequencing data, and is able to integrate the chimeras into the following bioinformatics analysis, rapidly. By comparison with previous pipelines, ChimeraMiner took less than a half the time (43.8% for MDA1 and 43.0% for MDA2) to accomplish the detection. It was mainly due to the optimization of detecting process and the different alignment tools selected. While searching for overlap sequences, ChimeraMiner used a solo cyclical alignment, but the previous pipeline utilized two nested loops. Beyond all doubt, one solo cyclical alignment took less time than the nested ones. Meanwhile, the alignment tool BWA-MEM was used in ChimeraMiner, and the previous pipeline used SOAP2. BWA-MEM was considered to take less time in paired-end reads alignment than SOAP2 does ,36,37. BThe possible generation mechanism of the two single-ended chimeras has been illustrated before in ,8, and tMDA is highly susceptible to contamination, especially when working with single cells. Chimera formation is thought to be a problem in single-cell MDA and is potentially arisen by strand switching during the displacement process. Herein, we successfully utilized ChimeraMiner to distinguish chimeras and normal reads in the sequencing data of single-cell MDA. To the best of our knowledge, there is no systematic discovery and analysis of chimeras in single-cell MDA sequencing data previously.After detecting chimeras in all samples, we statistically analyzed the different chimeras. First, we found that the conventional MDA (MDA1 and MDA2) generate more chimeras. It is mainly because the DNA used in the sample of MDA1 and MDA2 was lower than that in one cell, about 10% to 40% of a haploid genome. The limited amount of input DNA caused the increase of amplification duration, and the number of chimeras increased synchronously. Among single cell samples, inverted chimera was remaining the dominant type of the single-end chimera , which iBy comparing the quantity of SVs between amplified and non-amplified genomic DNA, we known that the chimera introduced many false positive SVs. After filtering the detected chimeras, 83.82% of the false positive SVs were removed. The results indicated that the impact of chimeras on SV detection is massive, and the identification of chimeras from sequencing data, enables more accurately recognized SV. A practical SV identification pipeline is desired to be developed, based on the accurate recognition of chimeras. The human genome sequence  was downWhole genome sequencing data, from MDA and single cell MDA, were downloaded in FASTQ format  from NCBThe hardware of the computing platform was four Xeon E7-4820 CPUs, 256 giga-base ram and the operating system is Ubuntu 14.04.5. In comparison, we respectively used twelve threads in the mapping step and one thread in the script of detecting chimera. The RAM used by ChimeraMiner is relatively small because of all scripts written in Perl5.18 .Using the pipeline of previous studies as guideline, ChimeraMiner was composed of three parts: (A) sequencing reads pre-processing, reads alignment, and filtrating; (B) unscrambling of the soft-clipped alignment reads and realignment; (C) searching overlap and achieving valid chimeras.Firstly, we achieved clean reads by removing the raw reads with N base. The clean data was mapped to the human reference genome hg19 by using BWA alignment software with BWA-MEM algorithm in paired-end mode. The alignment results were saved as SAM format  and the In this step, we split the soft-clipped alignment reads retained in accordance with the soft-clipped alignment locations and strand orientation, and reconstructed new paired-end files, using the parts of all soft-clipped alignment reads. According to the length of the chromosome, we filtered the soft-clipped sequences whose subsections were shorter than the minimum length, which was calculated by the following equation:For example, the length of chromosome 1 is 249,250,621 bp; the minimum length of the soft-clipped sequences, calculated by the equation above, is 14. The soft-clipped sequences, which contained subsections shorter than a 14 bp trend to be multi-aligned on chromosome 1. Therefore, these soft-clipped sequences are removed. Meanwhile, any soft-clipped sequences which contained subsections shorter than 8 bp were removed due to the large probability of multiple alignment. Then, the reconstructed data were mapped to hg19 with BWA-backtrack in paired-end mode . Figure. The forIn this step, we aimed to search the overlap sequences of split reads, and achieve the discovery of all the valid single-end chimeras. Firstly, according to the results of realignment, the recordings of the two segments mapped to different strands of the chromosome were regarded as potential inverted chimeras, and those mapped to the same strands of the chromosome, were regarded as potential direct chimeras. Then, we designed an algorithm for searching the overlap sequences of the potential chimeras. Firstly, we assumed that the overlap sequence located at the end of former subsection, and restricted the length of the overlap sequence with the maximum 30 bp. To search the overlap sequence, the last 31-nt of the former subsection were cut. If the length of the former subsection was shorter than 31-nt, we polished the subsection with N forward. Then, based on the mapped position and orientation of the following subsections, we reverse-extended the following subsection by 31-nt, from the coordinate of its first base on the reference genome. The last 31-nt of the former subsection was cyclically aligned, with the extended 31-nt of the following subsections, by shortening one nucleotide in one loop, until the overlap sequence was found or no sequence was left. We tolerated one mismatch during the cyclically alignment, as the mismatch cannot appear at the beginning of the overlap sequence a. HereafWe constructed ChimeraMiner, an improved chimeric sequence detection pipeline, based on BWA-MEM for analyzing the sequencing data of MDA, and classified the chimeric sequences into three types: Direct chimera, inverted chimera, and insertion chimera. ChimeraMiner accurately classified 24,375,539 (about 6.31%) read pairs from MDA1 and 17,480,982 (about 6.62%) read pairs from MDA2 as chimeras, spending the previous pipeline, ChimeraMiner screened 6,736,168 novel chimeras, most of which have been demonstrated to be missed by the previous pipeline. Furthermore, by applying ChimeraMiner in several single cell sequencing data, all types of chimeras were discovered in every single-cell dataset. MDA introduced plenty of false positive SVs into single cell sequencing data, and the identification and filtration of chimeras were able to remove most of the false positive SVs. ChimeraMiner revealed advanced efficiency in discovering chimeric sequences, and is promising to be widely used in single-cell sequencing to reduce the influence of chimeras to single-cell studies.


Transposons and other repetitive sequences make up a large part of complex genomes. Repetitive sequences can be co-opted into a variety of functions and thus provide a source for evolutionary novelty. However, comprehensively detecting ancestral repeats that align between species is difficult because considering all repeat-overlapping seeds in alignment methods that rely on the seed-and-extend heuristic results in prohibitively high runtimes.Here, we show that ignoring repeat-overlapping alignment seeds when aligning entire genomes misses numerous alignments between repetitive elements. We present a tool, RepeatFiller, that improves genome alignments by incorporating previously undetected local alignments between repetitive sequences. By applying RepeatFiller to genome alignments between human and 20 other representative mammals, we uncover between 22 and 84 Mb of previously undetected alignments that mostly overlap transposable elements. We further show that the increased alignment coverage improves the annotation of conserved non-exonic elements, both by discovering numerous novel transposon-derived elements that evolve under constraint and by removing thousands of elements that are not under constraint in placental mammals.https://github.com/hillerlab/GenomeAlignmentToolsRepeatFiller contributes to comprehensively aligning repetitive genomic regions, which facilitates studying transposon co-option and genome evolution. Source code:  A substantial portion of vertebrate genomes consist of transposons and other repetitive sequences , 2. WhilThe nature of repetitive sequences such as transposons, however, leads to many paralogous alignments, which pose a challenge for comprehensively aligning orthologous repeats between vertebrate genomes. Most methods for aligning entire genomes use a seed-and-extend heuristic, originally implemented in BLAST , to findHere, we investigated to what extent aligning repetitive sequences are missed in whole-genome alignments. We show that ignoring repeat-overlapping seeds misses between 22 and 84 Mb of mostly repetitive elements that actually align between mammals and we provide a tool, called RepeatFiller, to incorporate such repeat-overlapping alignments into genome alignments. We further show that a subset of aligning sequences detected by RepeatFiller evolve under evolutionary constraint, which uncovers previously unknown conserved non-exonic elements (CNEs) and thus improves the annotation of constrained elements.To investigate how many aligning repetitive elements have been missed in alignments between mammalian genomes, we adopted a previously developed approach that was initially devised to detect novel local alignments between a pair of distantly related species , 25. TheTo investigate how many aligning repetitive elements can be added by RepeatFiller, we built alignment chains between the human (hg38) genome assembly and the genomes of 20 other mammals that represent the major mammalian clades Fig. . We founP = 0.0004, P = 0.067, Next, we investigated what factors are associated with differences in the amount of newly aligned sequences per species. In these tests, we excluded rhesus macaque, which is closely related to human, as an outlier. First, as expected, the percent of the genome that is repeat-masked significantly influences the number of newly aligned bases . For birds and insects, whose genomes generally consist of <20% repeats [D. melanogaster genome) , reptiles , and insects  Fig. . For repme) Fig. . Thus, dNext, we investigated whether some of the newly aligning sequences show evidence of evolutionary constraint, which indicates purifying selection and a biological function. To this end, we used the pairwise alignments, generated either with or without RepeatFiller, to build 2 human-referenced multiple genome alignments of 21 mammals with Multiz . Then, wcis-regulatory elements [P < E−16, P < E−16, Dividing the conserved regions detected in the alignment without RepeatFiller into exonic and non-exonic regions, we found that 99.8% of the exonic and 97.4% of the non-exonic regions are also classified as constrained in the RepeatFiller-subjected alignment. Because conserved exonic regions are virtually identical, likely because they rarely overlap repeats, we focused our comparison on the conserved non-exonic elements (CNEs), which often overlap elements . This coMEIS3, a homeobox transcription factor gene that synergizes with Hox genes and is required for hindbrain development and survival of pancreatic β-cells [MEIS3  to the query species. We used the RepeatMasker repeat annotation for hg38, available at the UCSC Genome Browser , to deteBefore building multiple alignment, we filtered out low-scoring chains and nets, requiring a minimum score of 100,000. We used Multiz-tba  with defTo identify constrained elements, one needs a tree with branch lengths representing the number of substitutions per neutral site. We used 4-fold degenerated codon sites based on the human Ensembl gene annotation to estimate the neutral branch lengths with PhyloFit . To idenRRID:SCR_000563) [RRID:SCR_006646) [To compare constraint in genomic regions classified as constraint in only 1 alignment, we used GERP++ (_000563)  with def_006646)  and calcProject name: RepeatFillerhttps://github.com/hillerlab/GenomeAlignmentToolsProject home page: Programming languages: Perl and PythonOther requirements: LASTZLicense: MIT LicenseRRID:SCR_017414ELIXIR bio.tools registry: biotools: RepeatFillerhttps://bds.mpi-cbg.de/hillerlab/RepeatFiller/. The CNEs that differ between both alignments are available in Supplementary Tables 3 and 4. The RepeatFiller source code is available at https://github.com/hillerlab/GenomeAlignmentTools. Other supporting data and code snapshots are available from the GigaScience GigaDB repository [The multiple genome alignments generated with and without applying RepeatFiller and the respective PhastCons conserved elements are available at pository .Supplementary Figure 1: Properties of newly detected CNEs. We compared CNEs already detected in the non-RepeatFiller alignments (dark grey) to 30,167 novel CNEs that were only detected after applying RepeatFiller (orange). (A) Violin plots overlaid by box plots show that newly detected CNEs are significantly shorter  and lack very large CNEs . For visualization, the shown distributions are cut at 200 bp. (B) The size distribution of newly and already-detected CNEs is similar to a power law distribution. (C) Violin plots overlaid by box plots show the percent A+T bases per CNE. Newly and already-detected CNEs are more AT-rich than randomly selected, non-conserved genomic regions of the same size as the already-detected CNEs. We repeated the sampling of non-conserved genomic regions 10 times and found a highly significant difference in each case.P < E−16 in a 2-sided Wilcoxon rank sum test.***: Supplementary Figure 2: Comparing constraint in conserved non-exonic regions that were only classified as conserved in the alignment with or without RepeatFiller. (A) Conserved non-exonic elements (CNEs) obtained by PhastCons for alignments with and without RepeatFiller (represented by light grey boxes) are largely identical. However, some of the regions are annotated as conserved either only in the RepeatFiller-subjected alignment (‘gained’ regions—orange boxes) or only in the alignment without RepeatFiller (‘lost’ regions—dark grey boxes). (B) For each position in these variable CNEs, we calculate the number of rejected substitutions (RS) with GERP++, separately for the alignments with and without RepeatFiller. The illustration shows that RepeatFiller adds more aligning sequences (red font). GERP++ computes the number of substitutions expected under neutrality from a phylogenetic tree that is pruned to the aligning species. That means that branches leading to non-aligning species (dashed grey lines) are ignored when computing the number of expected neutral substitutions. The difference in rejected substitutions per alignment column .


Alignment of sequence families described by profiles provides a sensitive means for establishing homology between proteins and is important in protein evolutionary, structural, and functional studies. In the context of a steadily growing amount of sequence data, estimating the statistical significance of alignments, including profile-profile alignments, plays a key role in alignment-based homology search algorithms. Still, it is an open question as to what and whether one type of distribution governs profile-profile alignment score, especially when profile-profile substitution scores involve such terms as secondary structure predictions.% in the number of true positives and up to 61.8% in the number of high-quality alignments with respect to the previous version of the COMER method.This study presents a methodology for estimating the statistical significance of this type of alignments. The methodology rests on a new algorithm developed for generating random profiles such that their alignment scores are distributed similarly to those obtained for real unrelated profiles. We show that improvements in statistical accuracy and sensitivity and high-quality alignment rate result from statistically characterizing alignments by establishing the dependence of statistical parameters on various measures associated with both individual and pairwise profile characteristics. Implemented in the COMER software, the proposed methodology yielded an increase of up to 34.2The more accurate estimation of statistical significance is implemented in the COMER method, which is now more sensitive and provides an increased rate of high-quality profile-profile alignments. The results of the present study also suggest directions for future research.The online version of this article (10.1186/s12859-019-2913-3) contains supplementary material, which is available to authorized users. Establishing homology between proteins is essential in evolutionary and highly important in structural and functional studies of proteins  and theiIn the presence of a large and steadily growing amount of sequence data, estimating the statistical significance of alignments plays a prominent role in alignment-based homology search algorithms . For an S (nonlattice) for large sequence lengths m and n has been proved [The limiting distribution of the ungapped local alignment score n proved –7 to be n proved  1\documλ and K  are parameters calculable from the score matrix used to align sequences and satisfying the conditions of the negative expected score and existence of at least one positive score.where Ample empirical evidence, e.g., –13, has Importantly, factors, such as the length of sequences being compared , 15, 16 This study aims at characterizing the distribution of profile-profile alignment scores , 24 to iHere, we use the COMER method  as a meaMoreover, the distribution of profile-profile alignment scores strongly depends on the extent of (dis-)similarity between unrelated profiles, or the null model of random sequence families. Real sequences do not conform to the canonical (and simplest) model, in which the amino acids in the sequence are iid, and exhibit a more complex structure with short- and long-range amino acid correlations , 28–30. In this work, the null model of protein sequence family does not explicitly incorporate correlations between families and leaves the profile-profile scoring function unchanged. Instead, we randomly generate profiles with properties of real unrelated profiles and take into account the correlations between them by establishing the dependence of statistical parameters on the compositional similarity between the profiles. The null model affects the probability of a score of a pair of profile positions, yet the change in composition does not alter the type of alignment score distribution but values of statistical parameters. The approach proposed here predicts differences in these values.Overall, this work purposes a procedure for estimating the statistical significance of profile-profile alignments, regarding the effects and factors that influence alignment score distribution, such as edge effects, compositional similarity, and profile attributes (length and the effective number of observations). The procedure builds on an algorithm proposed for generating profiles that resemble real sequence families. The results of the implemented procedure give rise to an analysis of its impact on sensitivity and alignment quality, which we also provide in this paper.The concept of estimating the statistical significance of profile-profile alignments, as proposed here, is illustrated in Fig. l1 and effective number of observations (ENO) n1 and another profile of length l2 and ENO n2 follows from combining the significance of the alignment score and that of the number of positive substitution scores in the alignment. The distribution of alignment scores depends on both the lengths of profiles being compared and their ENOs [The significance of the alignment between a profile of length eir ENOs   their length and ENO and (2) length and mutual compositional similarity provides two categories of profiles. Two categories of distributions of alignment scores, illustrated in Fig.  their leλu≡λ of the limiting distribution of sequence alignment scores  is the probability of sk, will decrease as the number of positive substitution scores increases (p(sk) increases for all k:sk>0). Therefore, low values of λu may indicate an increased probability for a high-scoring alignment to occur by chance due to compositionally biased regions in two sequences.where {λu calculated for a pair of profiles [The parameter profiles , 21 has We analyze the distribution of alignment scores of real unrelated profiles  for two reasons. One is to determine the type of distribution that describes them. The other reason is that these data provide a reference point for generating random profiles whose alignment scores would follow the same type of distribution and be similarly distributed.We found that alignment scores of real unrelated profiles follow an EVD and 85% of goodness-of-fit tests do not reject the null hypothesis. Supplementary Sections S2.3 and S3.1 in Additional file It is impractical to use real unrelated profiles to produce sufficient data for any values of profile length, ENO, and compositional similarity. Instead, random profiles are generated.It has been shown that using a null model to generate realistic random profiles improves remote homology detection . HoweverWe achieve that by using a modest number of “seed” profiles constructed for a diverse set of real sequences. Random profiles then result from adding noise to profiles/MSAs generated using these seed profiles as a model. Seeds provide properties of real profiles, whereas noise and randomization represent a means of controlling correlation between random profiles.An important feature of the algorithm that we propose (Algorithm 1) is that random profiles result from concatenating fixed-length fragments sampled randomly from a set of generated MSAs regardless of SS predictions the fragments entail. Supplementary Section S4.1 in Additional file s and the level r of noise added to S×M generated MSAs are the parameters that determine the degree of similarity among resulting random profiles. Too large s and/or too small r lead to highly correlated random profiles, while too small s or too large r result in divergent profiles and useless alignment statistics.Algorithm 1 shows that the fragment length s and r using two criteria: (1) the goodness of fit of the EVD to the empirical distribution of alignment scores and (2) the distance between the distribution function obtained for real unrelated profiles and the distribution function obtained from simulations. In this way, profiles generated using the optimal s and r possess features characteristic to real profiles and correlations between them do not dominate.Based on the results obtained for real unrelated profiles, we find the optimal ADup [D.We calculate the supremum class upper tail Anderson-Darling statistic ADup   and compositional similarity.p-value of each alignment of real unrelated profiles.Predict the statistical parameters and calculate the D between the empirical distribution function obtained for real unrelated profiles and the distribution function obtained in step 7.For each combination of pair values of profile ENO and length, calculate the distance s and r with respect to D have been found.Repeat steps 1–8 until the optimal The whole procedure for finding the optimal We discuss prediction of statistical parameters (step 7) below. Here we note that predictions are used to incorporate compositional similarity into the statistical model of profile-profile alignments.S=1012 seed profiles (M=1) suggest that considering a small number of different values suffices to find the optimal s and r: Large s increases correlation between profiles, whereas large r makes them unalignable. Figure s=9 and r=0.03.The results Fig. a obtaineSections S6.1 and S6.2  from aligning simulated profiles of length l1 and ENO n1 against profiles of length l2 and ENO n2 and (2) from aligning profiles of length l1 against profiles of length l2 with mutual compositional similarly λu, as shown in Fig. Let us consider two profiles, one of length n1,l1;n2,l2}  and the other for profiles characterized by the set of attributes {λu;l1;l2}. Assume that distributions A and B belong to the family of EVDs. Let Let A and B index two distributions. The first is obtained for profiles described by the set of attributes {and the corresponding conditional mean estimator of the scale parameter is a and b are parameters that depend on the parameters specifying the distributions of the location and scale parameters, respectively, in settings A and B  and (2) from aligning profiles of length l1∈L against profiles of length l2∈L with λu discretized. We denote the predictions of the location parameter in these two settings by Statistical significance for profiles of any length Statistical accuracy does not necessarily correlate with increased sensitivity , 22 and Observing a high correlation between the scale and the location parameters  on conditional mean estimates.Conditional mean estimators  and 4)  4) are ui,hc} in  and 6),,\documenmentclass2pt{minimThe number of positive profile-profile substitution scores in the alignment provides additional information to the alignment score. For example, the same alignment score can be the result of many weakly positive substitution scores or a few high scores. Therefore, it can be a useful indicator for estimating the significance of profile-profile alignments.ωn. It depends on the lengths of profiles being compared and its value  decreases as the alignment search space (the product of the profile lengths) increases: The distribution of the number of positive substitution scores can be accurately approximated by the negative binomial distribution (NBD) .) Let EN be the E-value of an alignment with score x obtained from searching a database of size lN. Let also E-value of an alignment with the same score x (and profile composition) obtained by searching a database of size EN and The expected number of local alignments with a score greater than or equal to ncreases , 12, 38.t}EN′ is  8\documE-value as the alternative model.We use this relationship when compensating for a limited number of randomly generated profiles used in simulations. We refer to the results obtained by calculating the corrected r=0.03 and a fragment length s=9, as determined in This section presents results from the application of the proposed procedure Fig.  for estiFor comparison, we have implemented and present results from the application of the method for estimating statistical significance proposed by  Supplem. We refep-value reported by COMER (or another tool) for their best match less than the specified p-value. The expected number of such queries is the product of the given p-value and the total number of queries. Therefore, the correspondence between the fraction of queries and the given p-value represents the theoretical result.We used COMER (or another tool) to search and align the profiles constructed for 5000 simulated Pfam  (v30.0) s=1,r=0.01) lead to overestimation of statistical significance. This result can be accounted for by unrealistic representation of protein sequence families (profiles).The results are shown in Fig. ωn statistic taken into consideration achieve comparable statistical accuracy, the sensitivity and high-quality alignment rate obtained using the latter model (w/o ωn) are lower. The same applies to the model implemented based on the previous research (S&G’08) (see below).Finally, although the models with and without the We compared the performance of the new COMER version with that of its previous version , where t4900 protein domains of the test dataset from the SCOPe database (v2.03) filtered to 20% sequence identity was used to evaluate performance. The test and training datasets shared no common folds.Profiles were constructed using two categories of MSAs. The MSAs for each domain sequence were obtained by running PSI-BLAST   that belonged to the same SCOPe superfamily or shared statistically significant structural similarity , Two evaluation modes were used to evaluate alignment quality. The local mode penalizes alignment overextension, whereas the global mode penalizes too short alignments . These evaluation modes were also used to evaluate the quality of maximally extended alignments produced by COMER and HHsearch (option -mact set to 0). The evaluation of maximally extended alignments reveals how the quality of local alignments changes with their extension. It provides an indication of the quality of the match between the query and a database protein, which is also important in protein homology modeling. displays a large decrease in sensitivity and HQA rate with respect to the previous COMER version v1.4) for PSI-BLAST MSAs. Among other differences from the new model, that model does not depend explicitly on the compositional similarity between profiles, which in part accounts for this decrease. Although version S&G’08 achieved similar statistical accuracy  and d2cfqa_ (f.38.1.2) are alpha-helical proteins but represent different SCOPe classes. They have different topology and do not share statistically significant structural similarity.λu=0.296 between the profiles constructed for the two domains similarity (distance) between unrelated sequence families strongly affects the distribution of scores of alignments between profiles that represent these families. The challenging aspects of characterizing the distribution of profile-profile alignment scores, however, are not limited to diversity across unrelated profiles. Similarity between SS, context and other predictions that accompany profiles complicate the characterization of alignment score distribution too.The importance of a null model of profiles motivated us to develop an algorithm for generating random profiles. According to the algorithm, profiles are generated by concatenating fixed-length fragments sampled randomly from real unrelated profiles (seeds) with added noise. Controlling the noise level and fragment length allows to produce random profiles that are neither overly divergent nor share an excessive similarity. In this way, random profiles possess features characteristic to real profiles, but correlations that do not allow them to be considered unrelated do not dominate. Alignments between randomly generated profiles allowed us to determine the dependence of statistical parameters on profile length and ENO and also on compositional similarity between profiles.The results suggest two important implications. First, profiles generated using long fragments (fragment length 9) represent real unrelated profiles much more accurately than do those obtained by randomly sampling profile columns. Statistics obtained from their alignments lead to higher statistical accuracy. On the other hand, randomly sampling of columns destroys higher-order dependencies inherent in real proteins and leads to the opposite result.Second, the compositional similarity between profiles has a large impact on estimating the statistical significance of alignments. Improvements in sensitivity and HQA rate can be accounted for in part by that a high compositional similarity may indicate that the proteins share common structural elements or the profiles involve false positives. Significance estimation dependent upon compositional similarity, therefore, has a positive effect.In fact, the issues of profile composition and random profile model are factors that hinder the application of techniques such as importance sampling  to accurBased on the above considerations and the results obtained by generating random profiles using one seed profile, we suspect that further improvements may result from introducing a new measure of profile “sequence affinity” (as opposed to the quantitative measure of the effective number of observations). The benefit may come from including into the model the statistical parameters dependent upon the new measure through, e.g., the application of the conditional mean estimator. The present study already demonstrated the conditional mean estimator to be both easily interpretable and effective when the calculation of compositional similarity or divergence between profiles was in use. And it provides possibilities for further improvements.We developed a methodology for estimating the statistical significance of profile-profile alignments such that improved statistical accuracy accompanies both increased sensitivity to homologous proteins and rate of high-quality alignments. The combination of statistics dependent upon different profile measures, an integral part of the methodology, may prove useful for future research, including developments of sensitive iterative search methods based on profile-profile comparison, where the importance of controlling false positives is particularly stressed.Additional file 1Supplementary Materials. Methodological details, derivations, evaluation description, additional simulation and application results. Additional file 2Figure S1. Distributions of alignment scores of real unrelated profiles for different pair values of profile ENO and length. (PDF 579 kb)Additional file 3λ and different values of length l. (PDF 467 kb)Figure S3. Distributions of alignment scores obtained from aligning pairs of real profiles with mutual compositional similarity Additional file 4Figure S8. Distributions of alignment scores of simulated profiles for different pair values of profile ENO and length. Additional file 5λ and different values of length l. Figure S9. Distributions of alignment scores obtained from aligning pairs of simulated profiles with mutual compositional similarity Additional file 6p-values corresponding to the empirical distribution function obtained for real unrelated profiles against estimated p-values using predicted values of the statistical parameters. (PDF 322 kb)Figure S10. Observed Additional file 7Figure S13. Distributions of the number of positive substitution scores observed in alignments of simulated profiles of different values of ENO and length. Additional file 8Table S1. Goodness of fit of the EVD to the distribution of alignment scores of real unrelated profiles. (PDF 35 kb)Additional file 9Table S2. Goodness of fit of the EVD to the distribution of alignment scores of real unrelated profiles. (PDF 35 kb)Additional file 10S=1012 seed profiles with s=9 and r=0.03. (PDF 65 kb)Table S3. Goodness of fit of the EVD to the distribution of alignment scores of profiles generated using Additional file 11S=1012 seed profiles with s=9 and r=0.03. (PDF 44 kb)Table S4. Goodness of fit of the EVD to the distribution of alignment scores of profiles generated using Additional file 12Table S5. Goodness of fit of the NBD to the distribution of the number of positive substitution scores observed in alignments of simulated profiles. (PDF 67 kb)


O(m) compared to O(m2). Comparing with state-of-the-art algorithms , SPARK-MSNA provided 50% improvement in memory utilization in processing human mitochondrial genome  with a better alignment accuracy in terms of average SP score and comparable execution time. The algorithm is implemented on big data framework Apache Spark in order to improve the scalability. The source code & test data are available at: https://sourceforge.net/projects/spark-msna/.Multiple sequence alignment (MSA) is an integral part of molecular biology. But handling massive number of large sequences is still a bottleneck for most of the state-of-the-art software tools. Knowledge driven algorithms utilizing features of input sequences, such as high similarity in case of DNA sequences, can help in improving the efficiency of DNA MSA to assist in phylogenetic tree construction, comparative genomics etc. This article showcases the benefit of utilizing similarity features while performing the alignment. The algorithm uses suffix tree for identifying common substrings and uses a modified Needleman-Wunsch algorithm for pairwise alignments. In order to improve the efficiency of pairwise alignments, a knowledge base is created and a supervised learning with nearest neighbor algorithm is used to guide the alignment. The algorithm provided linear complexity  It was an optimal sequence alignment algorithm with a tradeoff in computational time and space. For two sequences of length m and n, the time and space complexity is computed as O(m * n). By expanding the same algorithm for multiple sequence alignment (MSA), the complexity rises exponentially (O(mn) for n sequences of length m). Because of this high computational cost involved, NW algorithm cannot be used in multiple sequence alignment especially for large number of long sequences. Most popular implementations of MSA such as CLUSTAL2, MAFFT3, MUSCLE4 use approximation methods such as progressive and iterative approaches for faster execution and less memory utilization. Most of the algorithms implemented so far were derivatives of NW algorithm.Sequence alignment is used in bioinformatics to identify degree of similarity between biological sequences , in understanding functional, structural and evolutionary relationship between them. Sequence alignment is of vital importance in molecular biology for studies involving molecular function prediction, evolutionary tree reconstruction and disease analysis. Needleman-Wunsch(NW) algorithm5, are effective ways to process huge volume of data of the order of petabytes and more. Implementation using big data frameworks for sequence alignment/mapping were reported in the literature for instance, Sadasivam et al., 20106 presented a Hadoop based implementation of MSA using NW algorithm and Zhao et al., 20157 presented Spark based implementation of local alignment.The improvements in DNA sequencing technology has led to an unprecedented increase in the amount of DNA and genome data being available for studies. Therefore, it is important to improve the scalability and performance of MSA tools. Cloud computing and the recently emerged big data technologies such as the new programming paradigm called MapReduceet al.8 developed an algorithm which is proven to be highly efficient in performing MSA of similar DNA/RNA sequences. The algorithm uses centre star strategy along with trie tree data structure to improve the performance. Spark version of this algorithm HAlign II9, to support large volume of sequences reported promising results for similar DNA/RNA sequence alignment. MASC (Multiple Sequence Alignment Based on a Suffix Tree and Center-Star Strategy) is the implementation of same algorithm on CUDA architecture to obtain much faster performance for ultra large data sets10. PASTASPARK11 is another prominent implementation of MSA on Spark framework, which performs alignment based on SATé (Simultaneous Alignment and Tree estimation) and transitivity.Most of these research focused on improving the scalability of MSA using Big data frameworks, but not much research has happened in improving the MSA technique as such. These implementations are able to support large set of input sequences, but when it comes to massive DNA sequences, they are either unable to support or execute slowly when the count of sequences increase beyond 100. DNA sequences are highly similar compared to protein sequences. This similarity feature can be utilized to improve the alignment, and enable algorithm to tackle the volume and achieve better performance. Q. Zou 12 at the pairwise alignment level. DDGARM, an improved NW algorithm13 for pairwise alignment has proved that, for highly similar sequences, optimal alignment can be achieved by filling only 10% of the matrix. Our algorithm uses the concept of Q. Zou et al., on progressive alignment, with modified NW algorithm for improved pairwise alignment. Key characteristics of the proposed algorithm include, (a) Suffix tree data structure for storing input sequences and identifying common substrings between sequences, (b) A knowledge base and nearest neighbor learning layer to guide the pairwise alignment, (c) Modified Needleman-Wunsch algorithm to perform pairwise alignments at each stage in order to reduce the memory and execution time of alignments and (d) Parallelization using MapReduce method for suffix tree construction and pairwise alignment to further improve the execution time.MSA could be further enhanced with a bounded dynamic programming algorithm14. Hence it is chosen as the core of our algorithm. The basic flow of progressive strategy is to prepare a guide tree and use series of pairwise alignments to align the sequences based on the branching order in the guide tree. The guide tree is formed based on the pairwise distance of sequences. Guide tree is formed in the order from shortest to longest distant pair. Initially the most closely related sequence pair is aligned and then the remaining sequences are aligned to the previous alignment until all sequences are aligned. Pairwise alignment is performed at each stage and it is refined at the final step while summing up the alignments. In the refinement step, the early gaps are revisited to adjust the penalties based on aligned sequences from other pairwise alignments. There are many MSA algorithms which uses modified forms of progressive methods2. Details about progressive alignment method along with pseudo code is given in Supplementary Material . Suffix tree is known to provide optimal search time23, which means, identifying the node v, which is closest to the root for a given pattern P, such that P is a prefix of path(v) can be performed in time linear to the length of P. All leaves in the subtree of the node v then represent the occurrences of the pattern P in string S.Each input sequence is partitioned in to equal size segments and these segments are used to construct the suffix tree. Suffix tree is characterized by a root and each edge is labeled by the nucleotide in the sequence. For any node n DNA sequences with an average length of m, the time complexity for building a suffix tree for one sequence is O(m) (Lines 2–3 in Algorithm 1). After constructing the suffix tree, search the suffix tree for each segment of every sequence pair to identify the common substrings and matching segments. Searching the n sequences in the suffix tree costs O(nm) (Lines 5–7 in Algorithm 1). For the unmatched segments, record the percentage identity and difference in length if any. Since the sequences are partitioned to equal size segments, only the last segment of every sequence will be having different length. Only the unmatched segments are considered for pairwise alignments and the features ie; percentage identity and difference in length are used to extract learning from knowledge base. The guide tree for performing pairwise alignments are formed based on the similarity measure extracted for each sequence pair.If there are 13. The number of diagonals to be filled to obtain optimal alignment is not fixed in all cases. Hence, there is a need to find the minimum number of diagonals to be filled as a pre-requisite. This is done using dot plot approach. With some modifications to the dotlet24 algorithm, the number of diagonals to be filled can be obtained. Test results have proved that the similarity between sequences and the number of diagonals to be filled are inversely proportional. According to our previous research13, sequences with % identity (more than 50%) and difference in length (less than 25%) are reported to get a 50% improvement in memory utilization and execution time in pairwise alignments.Our previous research had proved that, for pairwise alignments, optimal alignment can be achieved by populating only limited number of diagonals of the matrixIn our approach, pairwise alignment is performed only for the unmatched segments. As the similarity between sequences increases, the number of segments to be aligned reduces. The modified alignment algorithm further reduces the complexity as similarity increases. The most distant segment pair from the input sequences are chosen to identify the number of diagonals to be filled. This improves the execution time and at the same time ensures that all pairwise alignments provide optimal alignment as it would be the highest of diagonals count for the given input set. Even though this step involves only one pairwise alignment, this could become costly for very large sequences. Hence, a knowledge base is built with training data and a learning layer with nearest neighbor algorithm is used to extract knowledge out of knowledge base. With more learnings, the knowledge base becomes more accurate and would result in faster learning.O(m2)for sequence segments having length m. In our modified alignment approach, the complexity reduces to O(m * k) + O(2m * d), where k is the difference in length and d is the number of diagonals filled. In case of highly similar sequences, k → 0 and O(m) compared to O(m2) in the case of traditional dynamic programming approach where we fill the entire matrix. The worst case complexity would be O(m2) as 2m * d becomes equal to m2, when the similarity between sequences decreases.In the traditional dynamic programming based pairwise alignment, the complexity is 13, it is known that the number of diagonals to be filled depends on the similarity level and difference in length. Prior knowledge about the number of diagonals to be filled is a pre-requisite for the pairwise alignment step. Using training dataset, a knowledge base is built with the mapping of sequence similarity to number of diagonals. Sequence similarity measure (percentage identify and difference in length) for the most distant segments are used for extracting the knowledge from knowledge base. Nearest neighbor algorithm is used to identify the best matching entry from the knowledge base26. Less number of dimensions (percentage identity and length difference) for pattern recognition was the driving behind selecting nearest neighbor as the learning algorithm. More details about nearest neighbor algorithm is given in Supplementary Material %). Count of diagonals will be fetched for this closest match and that will be used for the pairwise alignment in the progressive MSA. Each time a new set of features are encountered, for which a closest match does not exist in the knowledge base, dotlet algorithm is executed to identify the number of diagonals. This learning is then entered in to the knowledge base for future alignments. More entries in the knowledge base would improve the performance and accuracy of the alignment. Figure 27. More details about MapReduce programming model is given in Supplementary Material . Figure 29 dataset which is considered as the golden benchmark for MSA is more suitable for protein sequences and does not provide benchmark for large DNA sequences. Details about real-world applications of MSA involving large genome sequences are given in Supplementary Material  and 16 s rRNA. Dataset from PASTASPARK 200k RNASim is also included in the test data.Most of the MSA algorithms compare the test results with other commonly used tools/algorithms. Test results of SPARK-MSNA are compared with HAlign, HAlign II and PASTASPARK. MAFFT & MUSCLE are used for comparing results of small data sets. Even though MASC has reported highly competitive performance in handling large volume of data, the underlying architectures are different for both implementations, as MASC is implemented on CUDA processor. The algorithms HAlignThe human mitochondrial genome dataset is a sample for highly similar dataset. The dataset contains 672 human mitochondrial genomes with maximum length 16579 bp and minimum length 16556 bp. The percentage identity is >97% for this dataset. 200k RNASim dataset is used as dataset with moderate level of similarity with minimum sequence length as 748 and maximum sequence length as 1836. 16 s rRNA dataset is used for testing the performance on less similar sequence set. It has minimum length 807 bp and maximum length 1629 bp. Details of test datasets are provided in Table Mr is the length of the reference sequence, Si is the score of the ith column and Sri is the score of ith column in reference sequence. Si is calculated as normalized total score of column i, with pair value calculated as 2 if residues are aligned, 1 if one of the alignments has a gap and 0 otherwise. The test results show that SPARK-MSNA performs better in terms of memory utilization and accuracy, but has increased execution time compared to HAlign II. Progressive alignment and the refinement step increases the execution time, but that helps in achieving a better alignment. The reduced matrix alignment guided by knowledge base leads to reduction in memory utilization. MAFFT and MUSCLE provide better average SP score compared to SPARK-MSNA, but they are unable to handle large volume of dataset. MAFFT and MUSCLE failed to deal with datasets of size more than 1 GB. PASTASPARK is able to handle the large volume of data, but the execution time is much higher compared to HAlign II and SPARK-MSNA.Average SP score is used for comparing the alignment accuracy. SP score is calculated as the number of pairs of residues correctly aligned. The score is calculated as13. The trade-off is between execution time and alignment accuracy. SPARK-MSNA provides a better balance between the two by providing an optimum performance in terms of computational time and memory while retaining an average SP Score close to MAFFT.The modified Needleman-Wunsch algorithm for pairwise alignment of unmatched segments plays a crucial role in reducing the memory utilization for SPARK-MSNA. In the pairwise alignment step, HAlign II uses complete 2 × 2 matrix for alignment, whereas, SPARK-MSNA uses limited diagonals  to calculate the alignment. This modification provides up to 50% reduction in the memory utilized (depending on sequence similarity)The knowledge base guides the bounded dynamic programming for pairwise alignment. Hence, a rich knowledge base results in better performance and better accuracy. For highly similar sequences, the improvement is highly significant as very less number of matrix cells  are included in the alignment. As the similarity decreases, the number of matrix cells needed in the alignment increases and for highly different sequences, complete matrix is needed in the alignment, which makes it similar to normal dynamic programming. This is evident in the test result of 16 s rRNA dataset, where the memory utilization is similar to that of HAlign II.In order to test the efficiency of knowledge base, we added the knowledge base layer (training layer) to HAlign II and tested the same using mt. genome and 16 s rRNA datasets. Table O(m). For n sequences, the complexity is O(nm). The second stage is searching the suffix tree for all possible pair combinations of n DNA sequences. The search would incur a cost of O(m) for one sequence pair and we have nC2 pairs. So, the complexity of search becomes O(n2m). Building guide tree based on the similarity measure obtained from search is the next step. This incurs a linear cost of O(n). Due to less number of dimensions and cardinality involved in pattern matching, the complexity of learning layer becomes O(k).The most important feature of SPARK-MSNA is the improved time and space complexity. The first stage of SPARK-MSNA is construction of Suffix tree. The Ukkonen’s algorithm using MapReduce model is adapted here. For one DNA sequence of length m, the time complexity for building suffix tree is k is the segment length, x is the difference length of segments involved in pairwise alignment and d is the number of diagonals to be populated. For highly similar sequences, O(k). In traditional dynamic programming approach, the complexity is O(k2). The last step of summing up the alignment results to form the final alignment would incur a cost of O(nm). The learning step involves one pairwise alignment of the most distant segment pair and it incurs a cost of O(k2), if the learning is not available in the knowledge base.Pairwise alignment of unmatched segments of sequence pairs is the next step. Since we are adopting the modified Needleman-Wunsch algorithm, the complexity becomes O(m2) added to the complexity of the algorithm, where m is the sequence length.Building the knowledge base is not part of the main flow of the algorithm. It is part of the training phase and hence it does not add to the overall complexity of the algorithm. Whenever the appropriate learning is missing in the knowledge base, the learning step is implemented to enhance the knowledge base. Modified dotlet algorithm is performed to get the number of diagonals. The alignment is performed on the most distant segment of sequences and in such scenarios, there would be an additional O(m). As the similarity between sequences decreases, number of unmatched segments and the number of diagonals to be populated for alignment increases, this will make the worst case complexity as O(m2). Same is the case when learning step is involved.The overall complexity of SPARK-MSNA is In this work, we have focused on improving the efficiency of MSA involving large DNA sequences by utilizing its similarity feature and improving the performance with learning layer and parallel execution. The test results and complexity shows that, SPARK-MSNA provides a better trade-off compared to other MSA tools/algorithms in handling similar large scale DNA/RNA sequences. SPARK-MSNA provides better alignment and memory utilization with a comparable execution time with large sequences. In best case scenario, SPARK-MSNA reduces the memory utilization up to 50% along with better alignment compared to HAlign II. In worst case scenario, where we cannot reduce the number of matrix cells to be processed in the pairwise alignments, the complexity remains similar to HAlign II. Test results with learning layer added to centre star approach shows that a knowledge driven approach helps in improving the performance in terms of execution time and memory. Knowledge driven algorithms, which can learn from experience and use the learnings in future alignments are instrumental in handling large scale datasets.The proposed knowledge base uses only similarity feature for learning. Adding more features in knowledge base and alignment approach to utilize those additional features could provide a better result in future. RDD persistence using kyro serialization instead of raw data format for improved memory utilization is also planned as a future enhancement.Supplementary Material


Studies on the structures and functions of individual kinases have been used to understand the biological properties of other kinases that do not yet have experimental structures. The key factor in accurate inference by homology is an accurate sequence alignment. We present a parsimonious, structure-based multiple sequence alignment (MSA) of 497 human protein kinase domains excluding atypical kinases. The alignment is arranged in 17 blocks of conserved regions and unaligned blocks in between that contain insertions of varying lengths present in only a subset of kinases. The aligned blocks contain well-conserved elements of secondary structure and well-known functional motifs, such as the DFG and HRD motifs. From pairwise, all-against-all alignment of 272 human kinase structures, we estimate the accuracy of our MSA to be 97%. The remaining inaccuracy comes from a few structures with shifted elements of secondary structure, and from the boundaries of aligned and unaligned regions, where compromises need to be made to encompass the majority of kinases. A new phylogeny of the protein kinase domains in the human genome based on our alignment indicates that ten kinases previously labeled as “OTHER” can be confidently placed into the CAMK group. These kinases comprise the Aurora kinases, Polo kinases, and calcium/calmodulin-dependent kinase kinases. Mutations in kinases that lead to gain of function are frequently observed in many cancer types4, while mutations may also result in drug resistance rendering existing drugs inefficient3. Humans have over 500 genes that catalyze the phosphorylation of proteins, collectively called the ‘kinome’5.Protein kinases catalyze the transfer of a phosphoryl group from an ATP molecule to substrate proteins6. The active site is located between the two lobes where the activation and catalytic loops form the ATP and substrate binding sites.Protein kinase activity is found in a number of protein families and superfamilies in the human proteome. The vast majority of human kinases come from one very large, diverse family that share a common fold consisting of an N-terminal lobe, composed of five β-sheet strands and an α-helix called the C-helix, and a C-terminal lobe comprising six α-helices5 which they divided into 478 typical kinase genes  and 40 atypical kinase genes. The annotation of the human genome has improved since the Manning paper; currently Uniprot identifies 483 human proteins containing 496 typical kinase domains (https://www.uniprot.org/docs/pkinfam). Uniprot identifies 29 atypical human kinases. Some of these are distantly related to the typical kinase domain, thus making them a superfamily, including Alpha kinases7, ADCK kinases8, RIO kinases9, FAM20C kinases10, and the PI3-PI4 kinase family, which contains the protein kinases ATM, ATR, and MTOR11. In addition, there are proteins that do not appear to share an evolutionary relationship with typical kinases that also phosphorylate proteins, such as pyruvate dehydrogenase kinases12.In 2002, Manning and coworkers identified 518 kinase genes in the human kinome13 and structural and functional inferences14. In 2002, Manning et al. built a phylogenetic tree of 491 typical kinase domains from a multiple sequence alignment created without using any kind of structural information5. The accompanying poster and image of this tree is still widely used in scientific papers and presentations on kinases15. Multiple sequence alignments of kinases have been used to extend structural and functional information from the kinases with known structures to those without known structures. This includes the conformations of active and inactive kinases18, predictions of substrate specificity19, and analysis of kinase-drug interactions21.For any large protein family, an accurate multiple sequence alignment is the basis of an accurate phylogenyet al.5. The gappy regions are usually present between major elements of secondary structure, where the family members may have widely divergent sequence loop lengths due to numerous insertions that occurred in different lineages of family members. The gappiness makes alignments difficult to visualize and produces errors in phylogenetic inference22.A common problem in multiple sequence alignments of large, diverse protein families is that they are very ‘gappy,’ i.e., containing many gap characters in every sequence in order to align inserted segments of different lengths that may be present in only a small subset of the sequences. This is true of the alignment used to produce the kinome tree of Manning In this paper, we present a parsimonious, structure-based multiple sequence alignment (MSA) of 497 human typical kinase domains from 484 human genes. We developed the MSA from pairwise structure alignment to a reference kinase (Aurora A kinase) and sequence alignment for kinases of unknown structure to their closest homologues of known structure. One of our central goals was to develop a parsimonious alignment of kinases containing as few gap regions as possible. Our alignment therefore contains aligned blocks (in upper case letters) that represent common structural elements, usually of secondary structure elements and important motifs, present in most or all of the kinases. These regions are separated by left-justified, unaligned sequence regions in lower case letters containing insertions of different lengths present in only subsets of the kinases. Our alignment contains 17 aligned blocks and 16 unaligned regions between them.23. Finally, we used our alignment to produce an accurate phylogeny of the kinase domains. Guided by the phylogeny and HMMs for each group we assign a set of ten kinases previously categorized as “OTHER” by Manning et al. to the CAMK group, consisting of Aurora kinases, Polo-like kinases, and calcium/calmodulin-dependent kinase kinases.We validated this alignment with an all-against-all pairwise structure alignment of 272 protein kinases, and we show that the alignment is significantly more accurate than the Manning alignment and other recent alignments for classifying kinases24 (https://www.uniprot.org/docs/pkinfam), which divides ‘typical’ kinases into nine groups  and a group of OTHER kinases. It also contains a list of ‘atypical’ kinases divided into six families . To identify any kinases which are not included in the Uniprot list, we searched all human Uniprot sequences with PSI-BLAST and each human kinase domain as query. As a result, we were able to identify Uniprot entry PEAK3_HUMAN as an additional human kinase. It was identified in searches starting with PRAG1 and PEAK1. The relationship with protein kinases was confirmed with HHpred25.Kinase sequences were identified from the list of human kinases and pseudokinases provided by Uniprotet al. did not include any atypical kinase in their sequence alignment and phylogenetic tree. However, before discarding them from our dataset we wanted to examine the atypical kinase structures available from the PDB. Since Aurora A is a good representative of typical protein kinase domains, we have used its structure as a reference to identify structural similarities and differences between typical kinases and atypical kinases. The kinase domain of Aurora A has 251 amino acid residues with eight helices, seven β strands, and all the known conserved motifs without any unusually long insertions . The structure consists of 384 residues, 13 helices, and eight β sheet strands. Structure alignment with FATCAT26 aligned 192 residues with an RMSD of 3.92 Å, covering the N-terminal domain, the HRD and DFG motifs, and the E and F helices of the C-terminal domain. COQ8A’s N-terminal domain contains an additional subdomain of five alpha helices, three of which precede the typical kinase domain and two of which are inserted between beta strand B3 and the C-helix. Instead of the activation loop leading into the F-helix, the DFG motif leads into a bundle of four alpha helices that precede COQ8A’s F-helix, which is followed by one additional helix.Structure alignment with Aurora A indicated that four of the atypical kinase families are homologous to typical kinases Fig. —containi27) has been determined; only the N-terminal domain and the E helix could be aligned to AURKA with an RMSD of 5.8 Å over 120 residues. The remainder of the C-terminal domain of TRMP7 consists of two beta sheet strands, large coil regions, and a short helix. The human PI3/PI4 kinases consist of seven genes: ATM, ATR, MTOR, PIK3CA, PIK3CB, PIK3CD, PIK3CG, PRKDC, and SMG1. All of these except SMG1 have known structures (PIK3CB is represented by a structure of mouse PIK3CB). The structure of PIK3CA (PDB: 4L2Y28) aligns with Aurora A with RMSD 6.0 Å over 168 residues. The structures of two of the three RIO kinases  are known. The structures of RIOK1 (PDB: 4OTP29) and RIOK2 (PDB: 6FDN30) can be aligned with AURKA with 5.1 and 4.6 Å RMSD over 176 amino acids, because the orientation of the HG, HH, and HI helices are entirely different.There are six kinases in the Alpha-type kinase family: ALPK1, ALPK2, ALPK3, EEF2K, TRPM6, and TRPM7. The structure of mouse TRPM7 (PDB:1IAJ31 and PDK1 by 2Q8H32), and none of them resemble typical protein kinases. They consist of an N-terminal domain in the form of a bundle of four long α helices and a C-terminal domain of a five-stranded β sheet and three α helices. ECOD (Evolutionary Classification of Protein Domains) also does not classify these structures in the same homology group as the typical kinases33. While there is no structure of FAST kinase (Uniprot FASTK_HUMAN), the program HHpred25 found that the closest homologues in the PDB are restriction endonucleases , which do not appear to be homologous to typical kinases.Two of the atypical protein kinase families listed by Uniprot do not appear to be homologous to typical kinases. The PDK/PCKDK protein kinase family consists of BCKDK, PDK1, PDK2, PDK3, and PDK4. The structures of all of these proteins are known  in lower case letters, and left-justifies them. An example is shown in Fig. To examine the boundaries of conserved segments and common insertions across kinases, we first aligned the structure of Aurora A kinase N- and C-terminal domains separately to one representative structure of each of 271 other human kinases in the PDB using the program FATCAT37 which aligned the main conserved regions in a majority of the sequences up to the beginning of the activation loop. Because of very large insertions in the activation loop and in the C-terminal domain in some kinases, the C-terminal domain was aligned only within some families. For example, the AGC-family Great wall kinase (AGC_MASTL) has a 548 amino acid insertion in the activation loop that caused the entire AGC family to be misaligned in the C-terminal domain with respect to the other families.The creation of the MSA was a multi-step process. The initial alignment of all the kinase domain sequences was done using ClustalOmega38. As shown in Fig. The main conserved regions identified in Fig. Regions between the aligned blocks that contain variable sequence lengths were left unaligned, and edited in Jalview so that they were left-justified and denoted in lower-case letters. They were separated from the aligned blocks by two empty columns on each end. These blocks were made as short as possible while respecting the structure alignments.The sequences of kinases in each kinase group without known structure were aligned to their closest homologues of known structure, such that all sequence motifs were identified in each kinase. This was greatly facilitated by sorting the alignment by family  in Jalview, which was possible because of the naming convention described above indicating group membership  and sorting the alignment by these identifiers.39); the second kinase domain of OTHER_EIF2AK4 ; OTHER_PINK1 ; and OTHER_PAN3 .Some kinases without close homologues of known structure required pairwise sequence alignment to other individual kinases with PSI-BLAST and/or HHpred in order to identify regions of the kinase domain that occur after very long insertions or that are highly divergent from other kinases. A small number of kinases were not closely related to any mammalian kinase of known structure, and instead required a sequence alignment to a yeast or insect kinase of known structure, which could then be structurally aligned to AURKA. Examples include: OTHER_TP53RK . This segment remains part of the aligned region CL, since it occurs in only one kinase.Our final alignment of 497 kinase sequences consists of 2229 columns including all the residue positions and gaps. It has 17 aligned blocks and 16 unaligned low similarity regions. The 17 aligned blocks consist of 8 segments from the N-terminal domain  and 9 segments for the C-terminal domain . Each block is named for the main element of secondary structure that it contains, although they each contain adjacent loop regions. We name the lower-case insertion regions by the pieces of structure that they connect separated by a tilde, e.g. B2~B3 is the segment between B2 and B3. B1N and B1C are the N and C terminal segments of beta strand 1; some kinases such as STE_MAP3K8 have an insertion in the strand that produces an extrusion of the chain. CL is the catalytic loop that contains the HRD motif. This segment also contains B6 and B7, which are short beta strands in most kinases. It also contains a 5-residue insertion in one kinase, OTHER_POMK, which is evident in the structure of mouse POMK  is a short beta turn motif with consensus sequence PP[FY] between HF and HG that is conserved in most kinases both in sequence and in structure.The activation loop is divided into N and C terminal aligned blocks, named ALN and ALC, of 21 and 17 residues respectively. These are long enough to include the common phosphorylation sites of 65 tyrosine kinases (at positions 13 and 14 from the activation loop DFG-Asp) and 212 serine/threonine kinases (at position -12 from the APE-Glu residue of the activation loop C-terminus)A list of aligned and unaligned blocks is provided in Table 45 to visualize the conservation of residues in all the aligned blocks of our MSA . However, to determine the accuracy of our MSA we have compared it with the sequence alignments derived from all-against-all pairwise structure alignments of 272 human kinases in the PDB. Because changes in conformation of the activation loop or movement of the C-helix may affect the corresponding alignment, we used structures that carry an inward disposition of the C-helix as often as possible, as determined by our recent classification of the active and inactive states of kinases47: the sum of the number of residue pairs that are identically aligned in both the MSA and the structure alignments divided by the total number of unique residue pairs aligned in the MSA or the structure alignments or both (counting each pair only once). The Jaccard index shows the overlap between MSA and structural alignments, and penalizes both under- and over-prediction of aligned residues in the MSA.To perform the validation we have computed three quantities as described in Methods: (1) True positive rate (TPR): the number of residue pairs which are aligned in the MSA and also in pairwise structure alignments divided by the total number of residue pairs aligned in the structure alignments; (2) Positive predictive value (PPV): the number of residue pairs which are aligned in the MSA and also in the structure alignments divided by the total number of residue pairs aligned in the MSA; (3) The Jaccard similarity indexet al.5, Möbitz16, Kwon et al.23, a hidden Markov model (HMM) derived from our MSA, and the initial ClustalOmega alignment. The average TPR for our MSA is 0.97, which is significantly better than the Möbitz (0.88), Kwon (0.90), and Manning (0.80) MSAs. The initial ClustalOmega alignment had a TPR of only 0.74. Similarly, our MSA also has the highest PPV (0.97) and the highest value for the Jaccard index of 0.94. The HMM derived from our MSA has TPR of 0.92, which is less accurate than the MSA itself but more accurate than the Möbitz, Manning, and Kwon alignments. It may therefore be of use in aligning kinases from other species to our MSA of human kinases.The average values and distributions of these quantities are presented in Table We calculated the ‘gappiness’ of each element, which we identify as the average number of gap segments in each sequence in the MSA. These are also given in Table We examined the positions where the structure alignments and our MSA are discrepant Fig. . In some48), which is rotated by 100°, and thus the conserved glutamic acid residue and the entire C-helix do not align with the homologous residues in closely related kinases. An active form structure of STK38 is not available, but is predicted to have the C-helix salt bridge to the lysine residue in the β3 strand48.In the remaining cases, the discrepancies represent structural shifts of elements of secondary structure in some kinases relative to the kinase domain. This is particularly true of the C-helix, which shows alignment differences between the sequence alignment and the structural alignment at a rate of about 5%. Examination of cases where the C-helix is misaligned indicates that the sequence alignments are probably correct, and the differences with the structure alignment are because of shifts in the position or orientation of the C-helix relative to the rest of the N-terminal domain. For example, the C-helix is misaligned in 50% of the structure alignments of MAP2K1 and MAP2K2, despite the fact that the conserved C-helix glutamic acid that forms a salt bridge with a lysine residue in the B3 strand in most kinases is correctly aligned in our MSA. Another example is the C-helix of STK38  contains a 548-residue long insertion within the activation loop (residues 188–735) and is the longest kinase domain sequence at 801 amino acids. In the Manning and Möbitz alignments, the kinase domain is defined as residues 35–310, and both miss the entirety of the C-terminal domain that follows the activation loop. OTHER_RPS6KC1  has a 480-residue long insertion (residues 419–898) in the HD~HE unaligned region. In the Manning alignment, the kinase domain begins on residue 822; in our alignment it begins at residue 337. OTHER_CDC7 has a 98-residue long insertion (residues 441–539) between HG and HH, while the kinase domain is defined as residues 58–472 in the Kwon alignment, thus missing the HH and HI helices.49 and Hunter50 schemes. These groups consisted of AGC, CAMK, CK1, CMGC, NEK, RGC, STE, TKL and TYR. A total of 83 protein kinases were placed in OTHER category because no significant relationship to any of the nine groups was recognized. However, this classification was done with only a limited amount of structural information, and as shown above, the Manning multiple sequence alignment was only 80% correct. We have revisited the phylogeny and classification of kinases to see if we can assign groups to some of the OTHER kinases by benefiting from our structure guided multiple sequence alignment.In their paper on the human kinome, Manning et. al provided a phylogenetic tree and classified the human protein kinases into nine groups, extending the early Hanks51, we built the tree with the 17 conserved blocks from the alignment , MAP3K9 (MLK1), MAP3K10 (MLK2), MAP3K11 (MLK3), MAP3K12 (ZPK), MAP3K13 (LZK), MAP3K20 (ZAK), and MAP3K21 (MLK4) form a tight cluster in the TKL group branch. These were also in the TKL branch of the Manning tree. Similarly, six sequences consisting of the second domains of RPS6KA1, RPS6KA2, RPS6KA3, RPS6KA4, RPS6KA5, and RPS6KA6 which were annotated to be in the AGC group by Uniprot also cluster in CAMKs (as they are in the Manning tree). The first domains of these kinases are AGC members.et al. . Three kinases, CSNK2A1, CSNK2A2, and CSNK2A3, form a tight cluster within the CMGC group. These are listed as OTHER by Uniprot and while CSNK2A1 and CSNK2A2 were designated CMGCs by Manning. One kinase, PBK  is assigned to STE in Uniprot, but to OTHER by Manning. In our tree, it sits just outside the TKL group but we would still classify it as OTHER.Because of their remote homology, most of the OTHER kinases branch out into separate clades in between the major groups. Due to the smaller size of these clades and relatively low similarity between the members we have not classified them as individual kinase groups. In our tree there are seven OTHER kinases in Uniprot that are correctly assigned to groups by Manning. Four kinases–STK32A, STK32B, STK32C, and RSKR, form a branch within the AGC group. These kinases were also classified as AGCs by Manning 55. We have included it in the CAMK group because of its close sequence relationship with the other PLKs.We have identified a set of ten kinases from the OTHER category in both Manning and Uniprot that can be appropriately assigned to the CAMK group. They form a branch in the middle of the CAMK group by nine kinases consisting of AURKA, AURKB, AURKC, CAMKK1, CAMKK2, PLK1, PLK2, PLK3, and PLK4 Fig. . These ket al. We then scanned each of the 497 kinase sequences against the nine group HMM profiles. A cutoff score of 200 was consistent with the assignments by Manning except for the changes described above. The novel assignments are the ten kinases that we can confidently move from OTHER to CAMK described above. The HMM scores clearly assign them to CAMK rather than AGC or OTHER, since the new CAMKs cluster with the other CAMK kinases , positive predictive values (PPV), and the Jaccard similarity index. In a large-scale benchmark of sequence alignment methods, we referred to them as fD (for developer) and fM (for modeler) for TPR and PPV respectively57. Yona and Levitt subsequently used these values (renamed QD and QM) to benchmark profile-profile sequence alignments, and added QC or QCombined58, which is simply the Jaccard index. The Jaccard index penalizes both overprediction and underprediction in our sequence alignments. We used all three values for our alignment  to demonstrate that our alignment is more accurate than the others available.Alignments are only useful if they are accurate. While several multiple sequence alignments of human kinases have been published and are available onlineThe errors in our MSA of kinases are mostly limited to the boundaries of conserved blocks where the variability of residue positions across kinases make their unambiguous placement in aligned blocks difficult. However, structure alignments do not always align every homologous pair of residues in two proteins. This occurs when residues are disordered in one of the structures or where there is significant conformational change. In a small number of kinases the only structure available has a significantly rotated C-helix. Structure alignment therefore sometimes does not align the homologous residues of the C helix in two kinases. The same is true for the G-helix in some kinases, which may be positioned in different locations within the C-terminal domain, but retains a homologous sequence and structure, and thus is aligned differently in our MSA than in the structure alignments.60. The predictions which are close to the values from experimental structure suggest that the multiple sequence alignment is more accurate. However, when a large number of structures are available, a sequence alignment is better validated against structure alignments and the sequence alignments derived from them.While our MSA is guided by predicted secondary structure of kinases and benchmarked with pairwise structural alignments, there are other approaches like ConTest and QuanTest which conversely use prediction of contact maps and secondary structure to assess the quality of the alignmentet al. is based on an alignment that has TPR and PPV values of only 0.80 and 0.81. Uniprot also provides a classification of kinases into the same groups as Manning et al. (with the addition of NEK kinases). The Uniprot annotations that differ from Manning et al. are all incorrect. Fourteen kinases are placed in the wrong groups by Uniprot, and another seven are placed in OTHER but can easily be placed within one of the defined groups, in agreement with the Manning annotations. One kinase (PBK or TOPK) is labeled as STE by Uniprot but we agree with Manning’s annotation of OTHER.With a more accurate MSA in hand, we have revisited the phylogenetic tree of human kinases. The widely used phylogenetic tree of Manning et al. and Uniprot: AURKA, AURKB, AURKC, CAMKK1, CAMKK2, PLK1, PLK2, PLK3, PLK4, and PLK5. Manning et al. placed the CAMKK and PLK kinases adjacent to the CAMK group and the Aurora kinases at the base of the AGC branch (but did not designate them as AGC). From our hidden Markov model of CAMK kinases and a phylogenetic tree based on our MSA, these kinases fit clearly into the CAMK group. Experimental data confirm that these assignments are correct. CAMKK1 and CAMKK2  both phosphorylate Ca2+/calmodulin-dependent kinases and bind calmodulin63. There is also direct evidence of calmodulin binding to PLK164 and of a calmodulin homologue, calcium-and-integrin-binding protein (CIB), to both PLK2 (Snk in the kinome poster) and PLK3 (Fnk)65.Of greater interest, our phylogenetic tree and hidden Markov models for each group can help us assign ten kinases to the CAMK group; these are listed as OTHER by Manning et al. put the Aurora kinases at the base of the AGC branch (but not labeled AGC). The AGC kinases are closely related to the CAMK group. Both groups possess a B helix that is not present in the other families. The HMM and the phylogenetic tree both indicate that the three Aurora kinases fit better into the CAMK group than the AGC group. In an earlier study with colleagues, we have shown experimental evidence that Aurora A binds calmodulin66, supporting its assignment to the CAMK group. Calmodulin also binds to Aurora B kinase (AURKB), preventing its degradation via the E3 ligase FBXL2 subunit67.Manning Our MSA provides the benefit of a common numbering scheme using the columns of the alignment facilitating comparison across all the kinase sequences. The identification of equivalent residue positions helps in generalizing experimental data from one kinase to another.68. Creixell found that specificity could also be modulated by more remote sites19, based on a multiple sequence alignment of kinases derived with ClustalOmega37. It is likely that our more accurate alignment would facilitate this analysis and produce more reliable predictions. Other areas where an accurate alignment and phylogeny may be useful are in predicting inhibitor specificity69, regulatory mechanisms through protein-protein interactions, and computational protein design of kinases with altered functionality70.For example, substrate specificity is highly correlated with the amino acid type at a small number of positions within the substrate binding siteOur alignment is included as supplemental data and on our website, and will be updated as new structures are determined. We hope that it will be of use in kinase biology and therapeutic development.https://www.uniprot.org/docs/pkinfam). To identify any unlisted kinases we searched human sequences in Uniprot with PSI-BLAST using the typical and atypical protein kinase on the Uniprot page as queries. PSI-BLAST was also used to identify structures of human kinases or their closest homologues in the PDB. The structures of atypical kinases (or homologues thereof) were examined structurally using a stand-alone version of FATCAT provided by Adam Godzik  and structural superposition by CEalign in PyMOL to Aurora A. Four of the atypical kinase families are visibly related to typical kinases, but contain significant fold differences. The other two families are not homologous to typical kinases.The list of human typical and atypical protein kinases was obtained from Uniprot website (http://genenames.org)35. Each kinase sequence was labeled by group name underscore HGNC gene name, for example AGC_PRKACA for KAPCA_HUMAN. The 13 kinases that have two kinase domains in the polypeptide chain were labeled with an underscore, for example TYR_JAK1_1 and TYR_JAK1_2. The boundaries were determined with PSI-BLAST of the full-length Uniprot sequence against the PDB.A total of 497 typical kinase domain sequences from 484 kinase genes (13 genes have two kinase domains each) were used to create the MSA. These sequences were initially divided into 9 phylogenetic groups as per the Uniprot nomenclature: AGC, CAMK, CK1, CMGC, NEK, RGC, TKL, TYR, and STE, and a tenth group of diverse kinases designated OTHER. Gene names were retrieved from the Human Gene Nomenclature Committee website  and the program FATCAT71. However, for the kinases where a structure was not known, alignment of the closest known structures to Aurora A were used to edit the alignment with Jalview. In a few cases, the most closely related structures were not human or even mammalian kinases. In these cases, the non-human kinase was structurally aligned to Aurora A and the target kinase was added to the MSA by transitive alignment. For a few distant kinases where a closely related structure or sequence was not known, HHPred was used to identify similarity to another kinase72.The kinase sequences (except some with very long insertions like GWL) were aligned using ClustalOmega73) is completely disordered; the structure of PBK  contains two monomers with half of the N-terminal domains of each chain swapped with the other monomer, making it difficult to align to other kinase domain structures. The structure-based sequence alignments were created using FATCAT in rigid mode and optimized using SE36. For kinases with multiple structures known the structure for validation was selected based on their conformational states using our previously published nomenclature46. The active state BLAminus conformation was preferred over others, followed by different kinds of DFGin inactive states - ABAminus, BLAplus, BLBminus, BLBplus, BLBtrans and DFGout-BBAminus.The MSA was structurally validated using a set of pairwise structural alignments as a benchmark. The benchmark consists of all vs all pairwise structural alignments of 272 kinases with known structures in the PDB. Two kinases were excluded: the BUB1B kinase domain in the cryo-EM structure of the anaphase promoting complex . The TPR is the ratio of Ncorrect and the number of residue pairs aligned in the structure alignment (Nstruct). For computation of the TPR, residue pairs in the structure alignment are skipped if either or both residues are contained in the unaligned (lower-case) blocks of the sequence alignment. This takes care of situations that occur when two kinases have identical length segments between two of our aligned blocks; the structure alignment program would align them but they would be indicated as unaligned in our sequence alignment. The alignment of Kwon et al. also includes unaligned regions in lowercase and is treated in the same way.A residue pair between two kinases in the MSA was considered to be aligned if it was also aligned in the benchmark pairwise structural alignment of the same kinases. Using this information, the accuracy of the MSA was assessed by computing three quantities TPR, PPV and the Jaccard similarity index. For each pair of sequences, we first calculate the number of aligned residue pairs that are present in both the sequence alignment and the structure alignment . For the PPV, residue pairs in the aligned blocks of the sequence alignment are skipped if one or both residues are aligned to gap characters in the structure alignment. This is usually either because the residues are disordered (no coordinates) in one of the structures or because there is a significant conformational change of a loop and the residues are aligned to gaps. The Jaccard similarity index is the ratio of Ncorrect and the number of unique aligned pairs in either the structure alignment or the sequence alignment (counting each only once). For the Jaccard index, all the pairs skipped in TPR and PPV are also skipped. A script for calculating these values is available on https://github.com/DunbrackLab/Kinases. The aligned residues from the pairwise structure alignments are available on https://zenodo.org/record/3445533 (DOI 10.5281/zenodo.3445533).PPV is the ratio of We also compared our MSA accuracy with the previously published alignments. These alignments did not contain residue ranges in the Uniprot sequences, and used different nomenclature for the protein names. To identify a correspondence between the sequences in previously published alignments and our MSA, we performed PSI-BLAST searches of each of their sequences against Uniprot and renamed them according to our scheme (groupname_genename).52 was created which was used to create a phylogenetic tree with the neighbor-joining algorithm.The phylogenetic tree of human protein kinases was created from an MSA obtained by deleting the unaligned regions from our MSA. A distance matrix using the p-distance in the program MegaXhttps://github.com/evolbioinfo/goalign). The alignments were read by MegaX using the same algorithm as mentioned above to infer bootstrap trees. Finally, ‘transfer bootstrap’ values were computed using the stand-alone version of the program Booster (https://booster.pasteur.fr/)54.To perform bootstrap analysis on the phylogenetic tree we generated 5000 bootstrap alignments using the program goalign . It can be visualized in rooted and unrooted representations with bootstrap values using the buttons provided by iTOL interface.The tree was saved in Newick format and uploaded to iTOL webserver75 to create HMM profiles for each of the nine kinase groups as defined by Manning et al. The input MSA for each group was extracted from the main MSA and any empty columns were deleted. All 497 kinase sequences were run against the nine HMM profiles using the program hmmsearch75. The scores of each sequence against nine HMMs were sorted and the group with highest score against each sequence was identified.We used the hmmbuild program of the HMMER3 package76. Graphs were produced with the R Package77. The phylogenetic tree image was produced with the iTOL webserver53. Sequence logos were created using the webserver WebLogo (http://weblogo.threeplusone.com/)45.Protein structure images were produced with PyMOL v. 2.3.2Supplementary informationSupplementary information 2Supplementary information 3Supplementary information 4Supplementary information 5