For successful protein structure prediction by comparative modeling, in addition to identifying a good template protein with known structure, obtaining an accurate sequence alignment between a query protein and a template protein is critical. It has been known that the alignment accuracy can vary significantly depending on our choice of various alignment parameters such as gap opening penalty and gap extension penalty. Because the accuracy of sequence alignment is typically measured by comparing it with its corresponding structure alignment, there is no good way of evaluating alignment accuracy without knowing the structure of a query protein, which is obviously not available at the time of structure prediction. Moreover, there is no universal alignment parameter option that would always yield the optimal alignment.n is transformed into a (n + 1)-dimensional feature vector, then it is used as an input to predict the alignment quality by the trained SVR model. Performance of our work is evaluated by various measures including Pearson correlation coefficient between the observed and predicted MaxSub scores. Result shows high correlation coefficient of 0.945. For a pair of query and template, 48 alignments are generated by changing alignment options. Trained SVR models are then applied to predict the MaxSub scores of those and to select the best alignment option which is chosen specifically to the query-template pair. This adaptive selection procedure results in 7.4% improvement of MaxSub scores, compared to those when the single best parameter option is used for all query-template pairs.In this work, we develop a method to predict the quality of the alignment between a query and a template. We train the support vector regression (SVR) models to predict the MaxSub scores as a measure of alignment quality. The alignment between a query protein and a template of length The present work demonstrates that the alignment quality can be predicted with reasonable accuracy. Our method is useful not only for selecting the optimal alignment parameters for a chosen template based on predicted alignment quality, but also for filtering out problematic templates that are not suitable for structure prediction due to poor alignment accuracy. This is implemented as a part in FORECAST, the server for fold-recognition and is freely available on the web at  Ab initio folding method is based on physical principles and does not require prior knowledge on protein structures, but comparative modeling [As the number of protein sequences is exponentially growing, knowledge on their structures and functions is lagging far behind the growth rate of the number of new protein sequences because the experiments to determine structures and functions are difficult and time-consuming. One way to resolve this problem is computational methods such as structure and function prediction. In the case of protein structure prediction, computational methods fall into two categories; modeling  has showmodeling .The first step in comparative modeling is the fold recognition in which one searches for homologous proteins with known structure and chooses the best one that can be used as a template. After this process, the alignment between the selected template and the query protein is generated. Finally the alignment is used to build the 3-dimensional structure models by using 3D model building tools such as MODELLER ,3. High-Importance of alignment accuracy for comparative modeling has been already addressed . Among mTo overcome this problem, some methods such as Consensus , ESyPredWe introduce here a novel method not only to predict the alignment quality but also to improve the alignment quality by support vector regression (SVR) . Machinen is converted into a feature vector of length n + 1 composed of profile-profile alignment scores and the length of the query protein. The predicted MaxSub score is calculated by the SVR model specifically built for that template. The test results show highly accurate regression performance. For a pair of a query and a template, various alignments are generated by using many different combinations of alignment parameters. The SVR model for the template is then used to find the optimal alignment parameters which are specific to that pair. We name this method 'adaptive selection' method. The adaptive selection method outperforms the method which uses the universal alignment option for large-scale testing set.Our attempt to develop a method to predict the alignment quality is not entirely new. A related work  has beenn + 1) dimensional feature vectors which are input of SVRs where n is the length of the templates  which is given byyi is the predicted value, oi is the observed value, and N the total number of the predictions. The normalized MAE (NMAE) is defined as followswhere The last one is the root-mean-square error (RMSE) given byMAE, NMAE, and RMSE are 0.0775, 1.877, and 0.0969, respectively, also shown in Table The ultimate objective of predicting alignment quality is to find the best alignment. One straightforward, although not the best, way to do this is to choose a set of the optimal alignment parameters, such as gap opening penalty, gap extension penalty, baseline score, and the amount of secondary structure term, that would yield the best alignments overall. However, as seen in Table The results suggest the following alignment strategy. Instead of using single universal set of alignment parameters for all query-template pairs, by simply picking up a different set of the alignment parameters that are uniquely optimal for a query-template pair, the alignment can be improved. If we do so, as seen in Table Obviously, we do not know a priori which set of alignment parameters is optimal for a given query-template pair because the structure of a query protein is not known. Therefore, here we propose the 'adaptive selection' method. The adaptive selection procedure is carried out as follows. (1) Generate the alignments using many different combinations of alignment parameters. (2) Predict MaxSub scores of alignments using the trained SVR models. (3) Select the alignment that gives the highest predicted MaxSub score.-300 calculated by Wilcoxon signed rank test [When we follow the adaptive selection procedure, the average of actual MaxSub scores of the alignments selected by the adaptive selection procedure improves to 0.2563 (Table ank test ). It alsIn this section, we describe performance at three levels of SCOP hierarchy  to closely examine where the improvement is achieved. All the experiments carried out in the previous section are done for testing sets at the three different levels.The density diagram in Additional file In Table To check diversity of test set, sequence identities of query-template pairs are presented in Fig at each SCOP level, family Figure , superfaAll protein pairs in the testing set used in the experiments share the similar structure at least at the fold level (see Methods). It is, therefore, necessary to check whether the trained SVR models show reliable performance for proteins which do not share the same fold. In order to check this, 10 unrelated proteins per each template are randomly selected, aligned against the templates, and transformed into feature vectors. The vectors are then applied to SVR models of the templates to predict MaxSub scores. All the observed MaxSub scores are zero without exception. Thus all the predicted values should be zero in ideal situation. Histogram of predicted values is shown in Figure It is expected that two proteins in the same SCOP family have a similar 3D structure. There are, however, many alignments of the pairs in the same family for which observed MaxSub scores are zero . All MaxSub scores of the alignments generated by using all 48 options are zero. To check whether it is caused by the problem of profile-profile method, we perform the structural alignment by CE algorithm , and we Such alignments are tested by the fold recognition method developed in the previous study  to see tThe benchmark test of adaptive selection method is carried out on 62 targets of CASP7. We use EsyPred3D and Multiple Mapping Method (MMM) for the comparing. Both are publicly available web servers, and alignments and 3D models are provided. We used the default options of the servers.Out of all 88 targets of CASP7, 77 targets have significantly close template in SCOP 1.69 according to the result of fold search by Proteinsilico . The temTable In the process of protein sequence alignment, generally only one particular set of alignment parameters is used throughout the all protein pairs, regardless of their evolutionary relationship. In some cases, many alignments are generated using many different combinations of alignment parameters, and then the potentially optimal alignment is chosen purely based on experience or intuition. In this work, however, we select the alignment parameters which are predicted to give the highest MaxSub score specific to a pair of a query and a template. Our work is distinguishable to other efforts to improve the quality of protein sequence alignments in that we directly predict alignment quality with quite good accuracy. By predicting the alignment quality and then choosing the optimal alignment parameters based on the prediction, we show that the alignment quality can be improved significantly. Our method can be utilized to select not only the optimal alignment parameters for a chosen template but also good templates with which the structure of a query protein can be best predicted.n + 1)-dimensional feature vector. These feature vectors are used to train the SVR models for the templates. We rigorously test the performance of the method using various evaluation measures such as Pearson correlation coefficient, MAE, NMAE, and RMSE. Results show the high correlation coefficient of 0.945 and low prediction errors. Trained SVR models are then applied to select the best alignment option which is chosen specifically to the pair of a query and a template. This adaptive selection procedure results in 7.4% improvement of MaxSub scores, compared to the scores when single best option is used for the all query-template pairs.In summary, we develop a method to predict the MaxSub score as an alignment quality of a given profile-profile alignment between a query and a template. The alignment between a query protein and a template of length n is transformed into a , a positive score is added (subtracted) if the predicted secondary structure of the query protein at the position i is the same (different) type of secondary structure of the template protein at position j. Finally, the constant baseline score (b) is added to the alignment score.where The frequency matrices and PSSMs are generated by running PSI-BLAST  with defsai is the profile-profile alignment score at position i of a given template [query_length is the length of the query protein  Correlations between observed and predicted MaxSub scores at the family level. Adjacent color bar shows the mapping of relative density. (b) Plot of frequency distribution. (c) Plot of MAE distribution. (d) Plot of NMAE distributionClick here for filePerformance of SVR models at the superfamily level. (a) Correlations between observed and predicted MaxSub scores at the superfamily level. Adjacent color bar shows the mapping of relative density. (b) Plot of frequency distribution. (c) Plot of MAE distribution. (d) Plot of NMAE distributionClick here for filePerformance of SVR models at the fold level. (a) Correlations between observed and predicted MaxSub scores at the fold level. Adjacent color bar shows the mapping of relative density. (b) Plot of frequency distribution. (c) Plot of MAE distribution. (d) Plot of NMAE distribution.Click here for file


At intermediate stages of genome assembly projects, when a number of contigs have been generated and their validity needs to be verified, it is desirable to align these contigs to a reference genome when it is available. The interest is not to analyze a detailed alignment between a contig and the reference genome at the base level, but rather to have a rough estimate of where the contig aligns to the reference genome, specifically, by identifying the starting and ending positions of such a region. This information is very useful in ordering the contigs, facilitating post-assembly analysis such as gap closure and resolving repeats. There exist programs, such as BLAST and MUMmer, that can quickly align and identify high similarity segments between two sequences, which, when seen in a dot plot, tend to agglomerate along a diagonal but can also be disrupted by gaps or shifted away from the main diagonal due to mismatches between the contig and the reference. It is a tedious and practically impossible task to visually inspect the dot plot to identify the regions covered by a large number of contigs from sequence assembly projects. A forced global alignment between a contig and the reference is not only time consuming but often meaningless.We have developed an algorithm that uses the coordinates of all the exact matches or high similarity local alignments, clusters them with respect to the main diagonal in the dot plot using a weighted linear regression technique, and identifies the starting and ending coordinates of the region of interest.This algorithm complements existing pairwise sequence alignment packages by replacing the time-consuming seed extension phase with a weighted linear regression for the alignment seeds. It was experimentally shown that the gain in execution time can be outstanding without compromising the accuracy. This method should be of great utility to sequence assembly and genome comparison projects. Genome sequencing is the process to determine the exact sequential order in which a target organism's DNA is made up by the building blocks, called bases and abbreviated A, C, T, G. This process is carried out in two phases. The first phase is shotgun sequencing phase where the DNA molecule of the organism is randomly sheared into a large number of small fragments and the ends of the fragments are read base by base using a chemical procedure introduced by Sanger . The seci.e., the contigs of the target genome can be matched against the reference genome. The alignments of these contigs onto the reference genome can potentially produce an informative overall picture of the ordering of the contigs, and therefore greatly facilitate the scaffolding and gap closure. Using reference genomes in sequence assembly has become increasingly feasible as more genomes have been sequenced and therefore become available as potential references.A typical technique for ordering the contigs is called scaffolding which uses paired reads. It is worth noting that the scaffolding by paired reads does not guarantee a complete ordering . The method also assigns each alignment a score to assess its goodness so that non reliable contigs can be easily flagged for further analysis or even discarded directly. The method is based on clustering the seed alignments using a weighted linear regression, which is presented in the next two sections.In many cases, a visual assessment of the dot plot of the seeds of an alignment is enough to judge the reliability of a contig. However, more exact numerical figures are often needed to validate a contig. Besides, it is common to have sets of thousands (and even hundreds of thousands) of contigs to be matched against a reference, in which case the graphical method is ruled out. In this work, we propose a simple yet powerful algorithm that takes as input the set of exact matches  between a contig and a reference genome, and produces as output the starting and ending coordinates of the most likely global alignment that exists between the two sequences. In Figure wi  and a y-intercept bi. Our goal is to identify a strong alignment between the contig and reference, cluster the seeds that would belong to such an alignment, and filter out the "outliers"  of each match. It is reasonable to assign this weight to each seed, since the longer a match is, the more significant weight it should be given in determining the collinear axis.where Differentiating the metric with respect to which is made equal to zero to solve for y-intercepts, respectively. It is interesting to note that this solution is surprisingly simpler than that of a standard linear regression, which involves computation equivalent to matrix inversion. The implementation of Eq. (3) is straightforward, and the results have an accuracy of about 70% as compared to a large set of human curated alignments.Eq. (3) suggests that the optimal This initial idea served as an inspiration for an improved algorithm, more adapted to the nature of the problem. Because of the very nature of linear regression, be it a standard one or a modified one like ours, the solution is necessarily a kind of compromise, which means that it can be skewed if many small scattered outliers exist. To mitigate such a problem, a further modification to the algorithm is presented in the next section.i.e., matched segments) between a contig and a reference sequence. Each seed alignment is described by a 4-tuple , where sx and ex are the starting and ending coordinates on the reference sequence, and sy and ey are the starting and ending coordinates on the contig sequence. As mentioned before, the goal is to find the boundary coordinates  of the region in the reference (as x-axis) that more strongly aligns to the contig (as y-axis), as shown in Figure Following is a description in pseudo-code of the final algorithm that we developed to process the seed alignments ; and v, a window size given as a percentage of the contig length.Algorithm:Lf and Lr for matched segments on forward strand and on reverse strand respectively.1. Compute the accumulated lengths FOR EACH Si ∈ S   IF (slope(Si) > 0)   THEN Lf += ||Si||   ELSE Lr += ||Si||.   Lf and Lr.2. Decide on the dominant strand by the maximum of IF (Lf > Lr)   THEN DS = +1   ELSE DS = -1.   Y-axis and assign them each a weight λ defined as the total length of segments whose y-intercept is within a window of size 2v.3. Project the segments on the dominant strand onto the FOR EACH Si ∈ S   IF (slope(Si) = DS)      THEN λi = ∑j ||Sj||, for all segments Sj whose y-intercept bj satisfies bi - v ≤ bj ≤ bi + v.      i*, that has the maximum weight. That is to say, segment Si* has most long segments within the window of the given size centering at it.4. Find the segment, indexed as i* = arg maxi(λi) and b* = bi*.Si* as a centroid to cluster all segments whose y-intercept falls within the window of size 2v centering at b*. Find the indexes j* and k* for the two boundary segments of the region where the contig and the reference are strongly aligned.5. Use the segment Sj whose y-intercept bj ∈ ;   Sk whose y-intercept bk ∈ .   Output:x1, x2) = , where sx is the starting coordinate of segment Sj* and ex is the ending coordinate of segment Sk*, all with respect to the reference sequence.    = , where sy is the starting coordinate of segment Sj* and ey is the ending coordinate of segment Sk*, all with respect to the contig sequence.    as suggested in the pseudo code, step 3 runs effectively in linear time when the window size is kept small.As for the time complexity, after the Now we present a simple example to help illustrate the mechanics of the algorithm. Suppose MUMmer found six exact matches in the forward strand, as shown in Figure y-intercept and the weight vectors are found to be y-intercept 14 only match No. 3 is within its window range, hence a weight of 2 was assigned. For y-intercept 2, matches No. 1, 2, 5 and 6 are inside its window range, then a weight 4 + 8 + 2 + 3 = 17 was assigned. By finding the maximum of y-intercepts are inside the window range of Assume a window of size 4 for this example. Forward strand is chosen as the predominant strand of the alignment. Then the  2, 17, 2, 12. In O. sativa var. japonica. Our algorithm was implemented in a perl script that takes as input the alignments generated from MUMmer. The pipeline consists of two steps: running MUMmer to find the seed alignments, and clustering these seed alignments by the weighted linear regression method to identify the mapping region. MUMmer's parameters can be tuned up such that only a consistent sequence of seeds is found, avoiding the extension phase in MUMmer that tries to fill the gaps between seeds using Smith-Waterman algorithm (this phase is the most time consuming part of the whole MUMmer pipeline). Our perl script parses MUMmer output file, and for each contig in the set it produces the two boundary coordinates of the region in the reference where the contig most likely aligns. We also give a score (from 0 to 100) to each projected mapping coverage based on the statistics and aggregate length of the set of seeds that compose each alignment as compared to the actual contig's length.We present in this section alignment results from real assembly data. In our attempt to assemble a region of the genome of several rice species, sets of contigs of varied cardinality and average length are produced and aligned to an available reference genome, namely the sequence of On the other hand, it is also possible to force MUMmer to make aggressive efforts to fill all the gaps and give as a result one extended alignment between the contig and the reference. Notice this is the goal of the weighted linear regression algorithm, but as will be seen, MUMmer takes much longer while trying to align the gapped regions base by base.The first thing to decide before running the algorithm is the window size. When long contigs are being aligned, a considerable number of insertions, deletions and miss-matches are expected, so the width of the diagonal region of the clustered matches should be widen. It is for this reason that we choose the size of the window to be proportional to the contig's length. This size is input to the program as a percentage that can go from 1% (the window size is 1/100 of the contig's length) to 100% (the window size is the whole contig's length).kb, 10 kb, and 100 kb, with 100 contigs in each subset. Each contig was sprinkled randomly with regions of mutations, insertions and deletions according to a given percentage of the contig length. For instance, a Mut-Ins-Del percentage of 10% indicates that the initial contig was first contaminated randomly with regions of mutations whose accumulated length amounts to 10% the length of the contig. Then regions of insertions under the same principle were introduced, and likewise with deletion regions. Since the location of the contig in the reference is known a priori, after these contigs are aligned to the reference using MUMmer and the predominant alignment region is predicted using our proposed algorithm, we can measure the percentage of the overlap (POV) between the two regions . We did so for all window sizes from 1% to 20%, as shown in Figure We tested the performance of the algorithm as a function of window size (seen as percentage of contig's length) and the results were plotted in Figure As expected, when mutations, insertions and deletions are low, a small window size is sufficient to maximize the POV between real and predicted regions, and as they are increased, longer windows are needed to accurately cluster all the exact matches of the alignment. Notice that this window size is desired to remain as low as possible to improve the speed of the algorithm. Our analysis suggests that a window size of 12% works well in all cases.Table We presented a method to cluster exact matches of pairwise sequence alignments by weighted linear regression which can be of great utility for genome assembly projects. This algorithm complements existing pairwise sequence alignment packages by avoiding their seed extension phase and replacing it with a weighted linear regression in the seeds of the alignment. It was experimentally shown that the gain in execution time can be outstanding without necessarily compromising the accuracy..The perl program is available at LL conceived the idea of alignments clustering, designed and prototyped the weighted Pairwise linear regression. AJG designed and implemented the modified algorithm and the experiments. AJG drafted the manuscript, and LL provided critical edits and important intellectual content. Both authors have read and approved the final version of this manuscript.


Genome sequence alignments form the basis of much research. Genome alignment depends on various mundane but critical choices, such as how to mask repeats and which score parameters to use. Surprisingly, there has been no large-scale assessment of these choices using real genomic data. Moreover, rigorous procedures to control the rate of spurious alignment have not been employed.We have assessed 495 combinations of score parameters for alignment of animal, plant, and fungal genomes. As our gold-standard of accuracy, we used genome alignments implied by multiple alignments of proteins and of structural RNAs. We found the HOXD scoring schemes underlying alignments in the UCSC genome database to be far from optimal, and suggest better parameters. Higher values of the X-drop parameter are not always better. E-values accurately indicate the rate of spurious alignment, but only if tandem repeats are masked in a non-standard way. Finally, we show that γ-centroid (probabilistic) alignment can find highly reliable subsets of aligned bases.http://last.cbrc.jp/.These results enable more accurate genome alignment, with reliability measures for local alignments and for individual aligned bases. This study was made possible by our new software, LAST, which can align vertebrate genomes in a few hours  The authors of BLASTZ tested their score cutoff by aligning two genomes after reversing, but not complementing, one of them . Homologmelanogaster/ananassae). Often, higher E-value thresholds are used for genome alignment than would commonly be used for database searches (e.g. BLAST). This is reasonable because genome comparison produces many thousands of local alignments, and a few hundred or even a few thousand spurious alignments would only amount to a small fraction of these.We have calculated the E-values implicitly used for several alignments in the UCSC genome database  (AdditioAlu elements. Simple repeats cause spurious (i.e. non-homologous) alignments with high scores, but non-simple repeats do not, because e.g. every Alu is genuinely homologous to every other Alu. Non-simple repeats cause a different problem: too many alignments. In pursuit of accurate (homologous) alignment, we focus on simple repeats.There is a general awareness that repeat-masking is important for genome alignment, but the efficacy of repeat-masking methods has not been assessed in this context. "Repeats" can be categorized into two types: simple (low entropy) sequences such as ATATATATAT, and non-simple repeats such as Many BLAST-like alignment tools have a capability known as "soft masking". This means that masking is applied for the first phase of the algorithm, when initial matches are found, but not for the second phase, when alignments are extended from the initial matches. This promises the best of both worlds: avoid purely repetitive alignments, but allow repeats within larger alignments.The scoring matrix specifies a score for aligning every kind of base with every other kind of base. The simplest scoring matrix, which is actually quite good for DNA, is: +1 for all matches and -1 for all mismatches. Given a set of trusted alignments, a scoring matrix is often derived using log likelihood ratios . This isEffective gap costs for protein alignment have been studied empirically , but notBLAST and similar methods, including BLASTZ and LAST, have an important but rarely considered X-drop parameter.  When extending gapped alignments, these methods terminate the extension when the score drops by more than X below the maximum previously seen . This seSequence alignments are inherently uncertain . This unno rigorous assessment of DNA scoring schemes.Surprisingly, we could find only one publication that attempts to determine optimal score parameters for DNA alignment using actual genome data: that of Chiaromonte et al., which recommends the HOXD70 matrix . This maAlu elements to any non-primate mammalian sequence is a false orthology prediction" [Alus and rodent B1 elements both originate from retrotransposition of 7SL RNA, and are thus homologous [It is notoriously difficult to measure the accuracy of genome alignments . Some prdiction" . This ismologous .  genomes, these regions are the focus of many downstream studies that use genome alignments. Moreover, small genomes (e.g. yeasts) consist mostly of coding sequence, as do the alignable parts of large but distantly related genomes . Finally, we submit that there is little justification for genome alignment parameters used up till now, and a limited assessment is better than none.Alignment accuracy can be measured at two levels: correctness of whole  alignments, or correctness of aligned bases. For genome alignment, the latter is arguably more relevant. This is because many downstream analyses, such as RNA structure prediction or detection of positively selected sites, depend on the base-level accuracy of the alignments . In suchThis study does not address the problem of distinguishing orthologous from paralogous alignments. This important problem is very different from that of aligning homologous bases, and requires its own specialized methods.In this study we empirically assess many parameter choices on whole genome alignments of several organisms. We were able to perform many thousands of genome alignments only by using our new alignment software, LAST. Our results give a practical guide for choosing repeat masking strategy, substitution and gap costs, and X-drop parameter - along with empirical estimates of false and true positive rates. We find that the best parameters significantly outperform current standard practice, often decreasing false positives by a factor of two or more at the same true positive rate.We tested E-value calculations and repeat-masking by aligning two genomes after reversing (but not complementing) one of them. Reversed genomes are convenient for estimating the rate of spurious alignments , becauseC. elegans has a chance similarity to a similar-length reversed repeat unit from C. brenneri. Because these units are tandemly repeated, the total length of the similar sequence region is amplified - increasing the apparent statistical significance of the match. DustMasker and runnseg can detect short-period tandem repeats, but this example shows that it is important to detect longer-period repeats too.Suppression of spurious alignments requires careful masking of tandem repeats. For example, Figure We obtained the non-standard TRF parameters by trial and error: we simply lowered TRF's mismatch cost, gap cost, and score cutoff until it worked. It is likely that a more principled and effective repeat-masking method than this can be found.is typically used for genome comparison, presumably out of reluctance to prevent any sequences from being aligned. Table In our tests, soft-masking fails to eliminate spurious alignments, but soft-masking Here, we assess which score parameters give the most accurate alignments, with high sensitivity and specificity. In order to measure accuracy, we need "gold-standard" alignments. For gold-standards, we used genome alignments implied by multiple alignments of proteins in the TreeFam database and of structural RNAs in the Rfam database ,29. ThesTable We tested alignment accuracy using 97 combinations of score parameters, which we denote in the format: match score: transition cost: transversion cost: gap existence cost: gap extension cost, e.g. "2:1:2:16:1". We also tested the BLASTZ/UCSC scoring schemes: the HOXD70 and HOXD55 matrices with a gap existence cost of 400 and a gap extension cost of 30. Each scoring scheme was combined with five X-drop values, for a total of 495 parameter combinations. In all cases, we hard-masked all the genomes with TRF, and aligned them with LAST using score cutoffs corresponding to an E-value of 1; an arbitrary choice. All genomes except the yeasts were additionally soft-masked with WindowMasker, to avoid wasting time on non-simple repeats. All parameters and results are tabulated in Additional file melanogaster/yakuba, Additional file Our test results are shown in Figure The poor performance of HOXD55:400:30 is not surprising, since these parameters tend to produce alignments with large, random flanks [The HOXD55 matrix itself would no doubt perform better if combined with higher gap costs. Indeed, the parameter combination 4:1:4:28:1, which approximates HOXD55 with a large gap opening cost, produces more accurate alignments . This means that we aligned all pairs of bases whose estimated probability of aligning is greater than 50% ,31. GeneTaken together, these results allow us to align genomes with greater accuracy and better estimates of the error rates.all spurious alignments with E-value less than 10-4  sequences other than tandem repeats are rare. Our TRF masking protocol leaves room for improvement, e.g. it works less well than DustMasker for Arabidopsis/Rice Figure . NeverthThe scoring scheme 2:1:2:16:1 gives very accurate alignments of distantly-related protein-coding sequences, whereas 1:1:1:7:1 is a good, conservative all-round choice. The main difference from the UCSC HOXD schemes is a relatively larger gap existence cost. Kent et al. used high gap existence costs when aligning nematode genomes . Lunter Since our assessment of score parameters is restricted to protein- and RNA-coding regions, there remains much uncertainty about these parameters for genome alignment in general. We can at least say this: there is both statistical and now (limited) empirical evidence that HOXD55:400:30 is a poor choice, and, ironically, there is little reason to reject the simplest possible (+1/-1) scoring matrix.The scoring schemes tested here use small integers, compared to the HOXD matrices. Small integers have some practical advantages: they work better with several methods for calculating statistical parameters ,37,38, aPlasmodium falciparum.Another constraint we applied to our parameter search is that all matches were given the same score. One might expect better performance by tuning the match scores to reflect A+T content. However, to be effective, this adjustment must consider not only the overall A+T content of the concerned genomes, but also the A+T content of the target homologous sequences. Note that for many genomes the A+T content of the Rfam sequences exceeds 50%, while the A+T content of the TreeFam sequences falls below the 50% level  and affine gap penalties. Although more complex methods are available  They use explicit probabilistic models.ii) They use probabilistic algorithms such as posterior decoding, rather than Viterbi/maximum-score alignment.iii) They use more intricate models with more parameters.The first is not really a difference, because classic alignment is equivalent to Viterbi decoding with a probabilistic model (a pair hidden Markov model) . As for One limitation of these more complex alignment methods is that they seem to fit more easily with global than local alignment. In particular, it is unclear how to calculate E-values that discriminate  homologies from chance similarities. It is also not obvious how to adapt their parameter-training algorithms to local alignment: we tried to do this, with poor results (not shown). In fact, many of these methods use BLASTZ alignments as a starting point, so they directly depend on accurate classic alignments. We expect that both simple and complex genome alignment methods will be useful in future, as is the case for protein alignment.Some sophisticated methods attempt not merely pair-wise but multiple genome alignment ,41,42. TFinally, parametric alignment  has beenWe have conducted the first large-scale assessment of repeat masking strategy and genome alignment parameters using real genomes - producing a practical guide to alignment parameters. We have tested a sufficient number of parameter combinations and genome pairs so that our results will be relevant to most genome alignment tasks. With our results, researchers will not only be able to produce more accurate alignments than with previous standard practice, but they will also have a much better idea of the reliability they can expect from such alignments.http://genome.ucsc.edu/: hg18, galGal3, fr2, gasAcu1, ce6, caePb2 [E. coli: NC_000913.2. B. subtilis: NC_000964.2. S. pombe: NC_001326.1, NC_003421.2, NC_003423.3, NC_003424.3. A. thaliana: NC_003074.5, NC_003071.4, NC_003070.6, NC_003075.4, NC_003076.5. O. sativa: NC_008394.1, NC_008395.1, NC_008396.1, NC_008397.1, NC_008398.1, NC_008399.1, NC_008400.1, NC_008401.1, NC_008402.1, NC_008403.1, NC_008404.1, NC_008405.1. S. cerevisiae: NC_001133.7, NC_001134.7, NC_001135.4, NC_001136.8, NC_001137.2, NC_001138.4, NC_001139.8, NC_001140.5, NC_001141.1, NC_001142.7, NC_001143.7, NC_001144.4, NC_001145.2, NC_001146.6, NC_001147.5, NC_001148.3, NC_001224.1. For the chaining evaluation, we obtained human and mouse CD4 regions from GenBank: U47924.1 and AC002397.1.The following genome versions were obtained from UCSC , caePb2 . The rem, caePb2 . E. colihttp://www.ncbi.nlm.nih.gov/CBBresearch/Spouge/html.ncbi/index/software.html with options: -eps_lambda 0.01 -eps_K 0.05 -max_time 600 -max_mem 2000[E-values are related to alignment scores by the following equation: E-value = 2 mnK * exp(-λ * score) . Here, m_mem 2000. For thematch = 2 mismatch = 5 delta = 5 PM = 80 PI = 10 minscore = 30 maxperiod = 200 -R. For "TRFs" , for consistency with Chiaromonte et al. [For the chaining evaluation, we used RepeatMasker open-3.2.7 (default mode) with e et al. .Rfam.full from Rfam 9.0 and aa_full_align from TreeFam 6.0 [aa_full_align and aa_seq. We then mapped the sequences to their parent genomes using BLAT version 34, with -ooc = 11.ooc for Rfam, and -q = prot -t = dnax -tileSize = 8 for TreeFam [We used the alignment files eFam 6.0 ,29. We e TreeFam . We keptTrue positives are defined to be aligned bases in the LAST (or BLASTZ) genome alignment that are also aligned to each other according to the gold-standard. For Rfam, false positives are defined to be aligned bases in the LAST (or BLASTZ) genome alignment, that are both aligned according to the gold-standard, but not to each other.  For TreeFam, we defined false positives more narrowly: we only counted aligned bases from the same TreeFam family. This is because there are inter-family homologies that are not captured in the TreeFam alignments.aa_seed_align instead of aa_full_align: the results are very similar  with the following parameters: -c -m110 -s5G. "-m110" tells it to ignore every third base when finding initial matches: this makes it considerably more sensitive for protein-coding sequence, and somewhat more sensitive for noncoding sequence (compared to contiguous seeds). We ran lastal  with options -p, -a, -b, -e, and -x to set the score parameters. For soft-masking, we used lastal option -u to mask during gapless but not gapped extension. Finally, we set the gapless X-drop parameter  to: 10 * .We ran T = 2 M = 50. We also used options Q, O, E, L, and Y to set the score parameters. Finally, we set the gapless score cutoff (K) to L*3/5, for consistency with LAST. In the right-most panel in row 6 of Figure P = 0 and either C = 3 (no chaining) or C = 1 (chaining). We used eleven score cutoffs (K): 20 to 30 for the unit matrix, and 2000 to 3000 for HOXD70.We ran BLASTZ v7 with these options: MCF conceived of the study, performed the experiments, and helped to draft the manuscript. MH developed the probabilistic alignment methods and helped with the relevant sections of the manuscript. PH developed prototypes of LAST and helped to draft the manuscript. All authors read and approved the final manuscript.Supplementary document. Three tables and nine figures.Click here for fileSupplementary data. Score parameter combinations, their statistical parameters λ and K, and alignment accuracy results.Click here for fileLAST version 94. Source code and documentation.Click here for file


Linear motifs (LMs) are abundant short regulatory sites used for modulating the functions of many eukaryotic proteins. They play important roles in post-translational modification, cell compartment targeting, docking sites for regulatory complex assembly and protein processing and cleavage. Methods for LM detection are now being developed that are strongly dependent on scores for motif conservation in homologous proteins. However, most LMs are found in natively disordered polypeptide segments that evolve rapidly, unhindered by structural constraints on the sequence. These regions of modular proteins are difficult to align using classical multiple sequence alignment programs that are specifically optimised to align the globular domains. As a consequence, poor motif alignment quality is hindering efforts to detect new LMs.We have developed a new benchmark, as part of the BAliBASE suite, designed to assess the ability of standard multiple alignment methods to detect and align LMs. The reference alignments are organised into different test sets representing real alignment problems and contain examples of experimentally verified functional motifs, extracted from the Eukaryotic Linear Motif (ELM) database. The benchmark has been used to evaluate and compare a number of multiple alignment programs. With distantly related proteins, the worst alignment program correctly aligns 48% of LMs compared to 73% for the best program. However, the performance of all the programs is adversely affected by the introduction of other sequences containing false positive motifs. The ranking of the alignment programs based on LM alignment quality is similar to that observed when considering full-length protein alignments, however little correlation was observed between LM and overall alignment quality for individual alignment test cases.We have shown that none of the programs currently available is capable of reliably aligning LMs in distantly related sequences and we have highlighted a number of specific problems. The results of the tests suggest possible ways to improve program accuracy for difficult, divergent sequences. Many eukaryotic proteins have highly modular architectures. Multidomain proteins are usual for transmembrane receptors, signalling proteins, cytoskeletal proteins, chromatin proteins, transcription factors and so forth. As a consequence, many programs have been developed for the detection and alignment of protein domains. Online resources can now provide a good overview of the globular domain architecture of a polypeptide sequence and the functions these domains are likely to perform, e.g. Pfam , SMART , InterprUntil recently, and in stark contrast to protein domain discovery, the bioinformatics field has had a negligible impact on LM discovery: motif discovery is generally performed by low throughput experimental delineation of protein interaction segments. The central problem confounding computational methods has been the lack of significance of motif matches when searching sequence databases, making it impossible to confidently identify all the motifs present in a given protein sequence by simple sequence analysis tools. The majority of LMs are between 3 and 10 amino acids in length and most have one or more ambiguous (variable) or wildcard  residues. Their short and degenerate nature makes real LMs difficult to distinguish from the background distribution of randomly occurring false positive motifs. Nevertheless, efforts are now underway to develop bioinformatics tools that will contribute to the linear motif discovery problem. As a first step, it is necessary to catalogue linear motifs and particular instances that are known to be functional. Such data collections include the eukaryotic linear motif (ELM) resource  and ScanOne of the major limitations in predicting short linear motifs is the evaluation of the many potential motifs found in each protein, to distinguish between true functional sites and incorrect occurrences of a given pattern. In the worst case, there are motifs which have such low support and low information content as to be almost indistinguishable from random noise in most datasets, e.g. the PCSK cleavage site K/RR  which plIt follows therefore that, in order to exploit evolutionary information optimally, we need to construct multiple sequence alignments of the highest quality. LMs that occur in several different phyla should appear as short patches of conservation in this alignment. However, a large majority of LMs are found in the natively disordered regions  that areThe main objective of the work presented here is to provide a standard way to assess the ability of a multiple alignment program to correctly align the linear motifs occurring in a set of related sequences. However, if the multiple alignment is to be used in a subsequent motif discovery system, it is important that (i) the sequences containing the motif should be accurately aligned and (ii) the sequences that do not contain the motif should not be aligned in the corresponding region. To address these issues, we have developed a new Reference Set that has been incorporated in the BAliBASE benchmark suite . The benUsing the new BAliBASE Reference Set, we then evaluated the accuracy of the motif alignments obtained from a number of widely used or recently developed multiple alignment programs. The performance of the different programs was assessed by comparing the alignments constructed by each program with the reference alignments. We show that none of the programs currently available is capable of reliably aligning LMs in distantly related sequences and we highlight a number of specific problems. This will hopefully generate interest in developing new algorithms and should provide program developers with guidelines for future enhancements that will improve the quality of motif alignments.The new Reference Set in the BAliBASE benchmark contains multiple alignments of sequences with functional linear motifs (LMs). To avoid confusion, the term 'sequence' will be used to refer to the full-length protein, while a specific region of a protein sequence will be referred to as a 'segment' or a 'motif'. The alignments are constructed from a set of reference sequences with experimentally verified motifs, known as ELM motif instances, extracted from the ELM database . PutativFor each reference sequence containing an ELM motif instance, the first step involves the detection of similar sequences in the Uniprot database  and the -3 are initially selected.(i) The Uniprot database is searched using the BlastP program  with the(ii) The selected full-length sequences are aligned using the PipeAlign  protein (iii) The sequences in the MACS are then annotated using the MACSIMS information management system . MACSIMS(iv) Finally, the automatic multiple alignment is manually verified and refined to assure the accurate alignment of the potential motifs with the current motif instance. A number of factors were taken into account during the manual validation, including the MACSIMS functional annotations, the presence of known globular domains or functional sites or the presence of conserved sequence segments close to the instance LM. The globular domains were generally well aligned by the PipeAlign toolkit and the manual refinement was limited to the potential LMs, identified by the ELM regular expressions. Low complexity regions were not realigned, as these could not be validated in most cases. During the manual refinement, any sequences containing motifs that could not be validated unambiguously were removed from the multiple alignment. These included motifs occurring in unrelated proteins (i.e. sharing no globular domains with the query sequence) and sequences with multiple occurrences of a motif, where the correspondence with the ELM motif instance was uncertain.Each Primary multiple alignment contains the complete set of sequences detected by BlastP, that share some similarity with the query sequence. In order to construct benchmark test sets, sequences are selected that correspond to a certain number of criteria. The following protocol is used to identify suitable alignments and sequences:(i) In order to reduce the size of the test sets, redundant sequences sharing >80% residue identity that are easy to align, are removed in a clustering step, based on the multiple sequence alignment.(ii) True positive, false positive and false negative motifs are defined (see figure (iii) Primary alignments containing less than 5 sequences with true positive motifs were excluded from the benchmark.(iv) Finally, primary alignments with at least one sequence in common were merged together to form a single reference alignment. This step is necessary because the original set of query sequences contains a certain number of homologous sequences, resulting in an overlap between the corresponding sets of BlastP detected sequences.The reference primary alignments are then divided into 4 Subsets, designed to evaluate the accuracy of multiple alignment algorithms under different conditions. Subsets 1–4 are constructed by automatically selecting the sequences from the Primary multiple alignment that correspond to a number of specific criteria.(i) For Subset 1, only sequences with validated LMs  are selected. Additional "false positive" motifs that match the ELM regular expression may be present at other positions in the alignment, as long as the LM corresponding to the motif instance can be identified unambiguously. The subset is further organised into three different groups, according to sequence variability: <20% identity, 20–40% identity and 40–80% identity.(ii) Subsets 2–4 include the sequences in Subset 1, together with additional sequences that cannot be aligned in the region of the motif instance. These additional sequences may cause problems for automatic multiple alignment programs:-▪ Subset 2 includes additional sequences that share some homology with the reference sequence, but that have possible 'errors'  in the region of the motif instance. Although these 'error' sequences do not contain a validated LM, they may or may not contain false positive matches to the ELM regular expression elsewhere in the sequence.▪ Subset 3 includes additional sequences with false positive matches to the ELM regular expression, that are found at other positions in the alignment and do not correspond to the motif instance.▪ Subset 4 includes additional sequences that do not have any examples of the LM, either in the region of the motif instance or elsewhere in the alignment.(iii) For each alignment in the benchmark, the validated LMs aligned on the motif instance are characterised according to three different features:• Structural context (globular versus disordered regions). Globular segments were predicted using the IUPred program . IUPred • Motif conservation. Motifs are defined as 'conserved' if all the true positive sequences in the multiple alignment share at least one conserved residue. Motifs in which no residues are conserved are defined as 'variable' motifs. The conservation is calculated based only on the validated motifs present in the multiple alignment, and does not necessarily correspond to non-wildcard positions in the motif regular expression.• Motif length .To assess the accuracy of the programs evaluated in this study, we calculate a sum-of-pairs score (SPS) which estimates the accuracy of the alignment of the motifs. The SPS corresponds to the percentage of correctly aligned pairs of residues in the alignment produced by the program. Only the sequences with true positive or false negative motifs, which can be aligned unambiguously in the region of the reference motif, are taken into account. The alignment of the other sequences has no effect on the SPS quality score.th column in the alignment of the motif by Ai1, Ai2,...... AiN. For each pair of residues Aij and Aik, we then define pijk such that pijk = 1 if residues Aij and Aik are aligned with each other in the reference alignment, otherwise pijk = 0. The score Si for the ith column is defined as:Suppose we have a test alignment and that N sequences in the alignment contain true positive or false negative motifs. Suppose also that the reference motif corresponds to M columns in the alignment. We can designate the iThe SPS for the alignment is then:r is the number of columns corresponding to the reference motif and Sri is the score Si for the ith column in the reference alignment. The score thus includes all columns in the alignment that correspond to residues in the motif instance, including variable and wildcard positions. For variable length motifs, positions containing gaps in the reference alignment are ignored.where MEight different alignment programs were tested, representing the mostly widely used approaches for the construction of multiple alignments:1. ClustalW  .The first test is designed to study the effect of sequence similarity on motif alignment accuracy and is used as the basis for the subsequent tests. Figure Although Probcons has improved the alignment of motifs in highly divergent sequences, it takes longer to run than MAFFT or MUSCLE. As shown in Figure The similarity of the sequences is not the only factor affecting the accuracy of alignment of the LMs. As shown in figure In order to further investigate possible causes for this variability, the LMs in subset 1, category V11, were annotated and classified according to three different criteria that might have a potential effect on motif alignment accuracy. Pearson correlation coefficients , including protein interaction sites, cell compartment targeting signals, post-translational modification sites and cleavage sites, play an essential role in the correct functioning of many multidomain proteins. The multiple alignments, developed here and included in the new BAliBASE Reference Set 9, represent real test cases and contain examples of experimentally verified functional motifs, extracted from the ELM database. In this version of the benchmark, we only included LMs that could be accurately aligned using a manual validation process. Motifs that could not be aligned unambiguously were removed from the reference alignments. This led to a bias in the benchmark test sets towards the known linear motifs that occur in globular domains, which is clearly not representative of all possible linear motifs, many of which are found in disordered regions that are difficult (or impossible) to align based solely on sequence information. In this case, the incorporation of multiple sequence alignment methods will not improve the motif prediction accuracy and other information will be needed, such as 3D structure, taxonomic range or cellular context. Nevertheless, it should be possible in future versions of the benchmark to include examples of these more difficult motifs, whenever experimental or other evidence allows an objective assessment of the alignment.The number of sequences in each reference alignment was then augmented by identifying related proteins in the public databases containing putative functional sites. The alignment of these potential motifs was validated and refined manually to ensure the correct alignment of the conserved residues.An important feature of the benchmark is the organisation of the reference alignments into different subsets that represent typical scenarios and problems that occur when trying to align short linear motifs in the context of a global multiple alignment. Thus, Subset 1 provides sequence sets at different levels of residue similarity, ranging from conserved sequences sharing 40–80% residue identity to more divergent sequences with <20% residue identity. In these alignments, all the sequences contain validated LMs. Subsets 2–4 provide alignments that represent a control to avoid over-optimisation of multiple alignment algorithms when using the Reference Set as a benchmark. For example, in Subset 3, aligning the false positive matches on the ELM motif instance would lead to a misalignment of the globular regions in the protein sequences. Subsets 2 and 4 consist of alignments that include sequences that do not contain the ELM motif instance and alignment of these sequences on the ELM motif instance would lead to over-prediction in subsequent motif discovery systems.The assessment of alignment quality was based only on the sequences containing verified motifs and only took into account the alignment of the LMs, ignoring the globular domain regions and the less well conserved, disordered regions. Nevertheless, we have shown that the accuracy of the LM alignment depends critically on the overall similarity of the sequence set, with a significant loss of accuracy for the most divergent sequences sharing less than 20% residue identity. This is true for all the programs tested in this study, regardless of whether a global or local alignment algorithm is used. For sequences sharing more than 20% identity, all the programs tested are capable of correctly aligning on average 80% of the motifs. Below this level of identity, in the "twilight zone" of protein sequences , the aliMost of the modern multiple sequence alignment programs have not been optimised to align the full length, highly modular protein sequences which abound in the human and other eukaryotic proteomes. Instead, they have primarily been designed to align individual globular domains by reference to their 3D structure superpositions: i.e. relatively short and collinear sequence regions. We consider that there may be a number of different ways to improve alignment of motifs, by taking them into account either when developing new alignment algorithms and optimising parameters, or when performing the alignment. Hopefully, an in-depth investigation of the factors affecting LM alignment quality and a subsequent optimisation of the diverse parameters should improve the performance of the different alignment programs. In this case, the risk of overfitting to the motifs may be acceptable when motif detection is the prime consideration.We have shown that none of the algorithms tested here is capable of reliably aligning all the LMs in distantly related sequences. It should be possible in the future to improve the reliability of LM alignment, either by combining a number of different algorithmic approaches and selecting a consensus alignment, or by including other contextual information, such as 3D structure, taxonomy or cellular context. If the resulting co-operative, knowledge based systems are capable of identifying the globular domain organisation of the proteins, then it should be possible to apply dedicated algorithms within the disordered, inter-domain regions, that can be used for the detection and alignment of over-represented or conserved sequence motifs. The success of such an approach will clearly hinge on the pertinence of the scoring function used to distinguish real motifs from random noise.The use of more accurate multiple sequence alignments should in turn increase the precision of the new motif discovery systems now being developed, with subsequent consequences in a number of important applications. For example, since LMs represent sites of protein interaction and act as nodes in regulatory networks, cell signalling cannot be understood until we can accurately identify such functional motifs. Furthermore, linear motif targets are now considered to play a role in drug discovery , as exemThe authors declare that they have no competing interests.EP constructed the reference multiple alignments and contributed to their validation and analysis. CC conceived the initial idea together with JDT and TJG and contributed to the definition of the scoring methods. TJG and OP contributed to the formalization of the alignment program evaluation. JDT supervised and took part in all stages of the project. All authors read and approved the final manuscript.Legend to additional figures S1, S2 and additional tables S1-4. Tables contain Friedman rank sum tests for data shown in figures Click here for fileg denotes LMs found in a globular domain, while n denotes LMs found in a non-globular domain. Figure S2. Comparison of motif alignment accuracy (SPS score) versus overall quality of complete alignment (NorMD score) obtained by the different alignment programs for the different similarity categories in subset 1 .Figure S1. Comparison of SPS scores for LM alignment (y-axis) for each reference dataset (x-axis) in Subset 1, V11 (<20% identity). The scores obtained by the different programs are shown in different colours and the maximum score obtained for each reference dataset is indicated by a red circle. On the x-axis, Click here for file


However, the lack of formally well defined methods to identify randomness in sequence alignments has prevented a routine application of alignment masking. In this study, we compared the effects on tree reconstructions of the most commonly used profiling method (GBLOCKS) which uses a predefined set of rules in combination with alignment masking, with a new profiling approach (ALISCORE) based on Monte Carlo resampling within a sliding window, using different data sets and alignment methods. While the GBLOCKS approach excludes variable sections above a certain threshold which choice is left arbitrary, the ALISCORE algorithm is free of ALISCORE was successfully extended to amino acids using a proportional model and empirical substitution matrices to score randomness in multiple sequence alignments. A complex bootstrap resampling leads to an even distribution of scores of randomly similar sequences to assess randomness of the observed sequence similarity. Testing performance on real data, both masking methods, GBLOCKS and ALISCORE, helped to improve tree resolution. The sliding window approach was less sensitive to different alignments of identical data sets and performed equally well on all data sets. Concurrently, ALISCORE is capable of dealing with different substitution patterns and heterogeneous base composition. ALISCORE and the most relaxed GBLOCKS gap parameter setting performed best on all data sets. Correspondingly, Neighbor-Net analyses showed the most decrease in conflict.Alignment masking improves signal-to-noise ratio in multiple sequence alignments prior to phylogenetic reconstruction. Given the robust performance of alignment profiling, alignment masking should routinely be used to improve tree reconstructions. Parametric methods of alignment profiling can be easily extended to more complex likelihood based models of sequence evolution which opens the possibility of further improvements. Multiple sequence alignments are an essential prerequisite in alignment based phylogenetic reconstructions, because they establish fundamental homology assessments of primary sequence characters. In consequence, alignment errors can influence the correctness of tree reconstructions -3. To deTo improve the signal-to-noise ratio, a selection of unambiguous alignment sections can be used. It has been shown that a selection of unambiguously aligned sections, or alignment masking , improvehttp://aliscore.zfmk.de. In short, within a sliding window an expected similarity score of randomized sequences is generated using a simple match/mismatch scoring for nucleotide or an empirical scoring matrix for amino acid sequences (see Methods), actual base composition, and an adapted Poisson model of site mutation. The observed similarity score is subsequently compared with the expected range of similarity scores of randomized sequences. Like GBLOCKS it is independent of tree reconstruction methods, but also independent of a priori rating of sequence variation within a multiple sequence alignment. Because of its explicit use of, although rather simple, models of sequence evolution, ALISCORE can be called a parametric method of alignment masking.The recently introduced alternative profiling method, ALISCORE , identifIt has been demonstrated that both methods correctly identify randomness in sequence alignments, although to a very different extent -12. A coQ,  to score differences between amino acids, Qij. Pairs containing indels and any amino acid are defined by using the value of a comparison of stop codons and any amino acid defined within Q. The observed score within a window of pairwise comparisons is generated by summing scores of single site comparisons. Starting from a multiple sequence alignment of length L, sequence pairs  are selected for which the following procedure is executed: In a sliding window of size w at position k, a similarity score S(k) is calculated comparing positions (i(k), j(k)), ∀ k ∈ , using the following simple objective function:As for nucleotide sequences , ALISCORObserved scores are compared to a frequency distribution of scores of randomly similar amino acid sequences with length given by the window size. The generation of randomly similar sequences follows the Proportional model , which iPij (t) as the probability of change from amino acid i to j, πj the frequency of amino acid j, μ the instantaneous rate of change, and t the branch length/time. Different to the algorithm used with nucleotide sequences in which scores are adapted to varying base composition along sequences and among sequences, the frequency distribution of scores of randomly similar sequences is only produced once for amino acid data. The frequency distribution is generated by: 1) collecting frequencies of amino acids of the complete observed data set, 2) generating 100 bootstrap resamples of this amino acid frequency distribution and 100 delete-half bootstrap resamples of each of the 100 complete bootstrap resamples, and 3) by using these 10,000 delete-half bootstrap resamples to generate 1,000,000 scores of randomly similar amino acid sequences with length given by the window size. This complex resampling leads to an even distribution of scores of randomly similar sequences. The frequency distribution of randomly similar sequences is used to define a cutoff c(α = 0.95) to assess randomness of the observed sequence similarity within the sliding window. Matching indels are defined as Qij = c/w. The principle of the complete scoring process is described in [with ribed in .Compared to GBLOCKS, using ALISCORE resulted in the exclusion of fewer positions in most data sets Figure . GBLOCKSResulting ML trees and Neighbor-Net graphs were examined under two different aspects: 1) We compared trees of all unmasked alignments with trees of differently masked alignments per data set to analyze the influence of each masking method on data structure and presence/absence of selected clades Figure . 2) We cIn general, ALISCORE masked alignments resulted in consistent ML topologies among identical but differently aligned sequence data. The ALISCORE algorithm performed in most cases better or at least equal well than the best GBLOCKS settings , GBLOCKS). Application of GBLOCKS(none) yielded less congruent trees.While plants, fungi, metazoans, and included subtaxa were fully resolved in unmasked trees of data set mtI, sister group relationships between major clades  could not be resolved without alignment masking. If alignments were masked according to the ALISCORE profile, all ML trees showed a sister group relationship between fungi and metazoans. In the case of the T-COFFEE alignment, the Amoebozoa were placed as sister group to Fungi + Metazoa. The alignment masking of GBLOCKS and GBLOCKS led to comparatively resolved topologies. Masking the alignments with the GBLOCKS(none) option reduced signal in the data Figure . BootstrFor the mtII data set we were not able to recover monophyletic plants and Amoebozoa as sister group to Fungi + Metazoa. The sister group relationship between Fungi and Metazoa was fully resolved in all ALISCORE, GBLOCKS, and GBLOCKS(none) masked data sets. GBLOCKS(none) and GBLOCKS masked alignments supported in several instances implausible clades Figure .Bootstrap support values only marginally increased after alignment masking Table . NeighboAgain, ALISCORE and GBLOCKS masking improved tree-likeness of the 12S + 16S nucleotide alignments at the taxonomically ordinal level. ALISCORE outperformed GBLOCKS and GBLOCKS in all instances. GBLOCKS(none) clearly performed worst  option tended to result in suboptimal node resolution and support values Figure , 3. In tThe ALISCORE and GBLOCKS approach performed quite similar and best on all data sets. This demonstrates that even a predefined set of rules suffices to extract randomness within sequence alignments. Talavera & Castresana  showed tALISCORE optionally uses a BLOSUM62 or various PAM matrices to score differences between amino acid sequences, or a simple match/mismatch score for differences between nucleotide or amino acid sequences. It uses a simple modified Poisson model of character state change  approach is independent of It will be a matter of further analyses, whether an extension of the sliding window approach to more realistic likelihood models of change and Monte Carlo resampling will further improve the performance. However, it would be conceivable to implement a more explicit model based approach in GBLOCKS as well. The advantage of improved parameterizing GBLOCKS could be a significant gain in speed compared to the sliding window approach. The best approach should be the most efficient one in terms of computational time and increased reliability of trees, the latter one admittedly hard to assess.COII, COIII, ND2, ND3, ND4L, ND6) show high sequence variability compared to the less variable genes . The first mt data set (mtI) included protein sequences of all chosen mitochondrial genes of 17 taxa. The second mt data set (mtII) comprised the five less variable genes out of data set mtI but with 24 taxa, corresponding to Talavera & Castresana [We used four different types of real data sets in combination with different alignment approaches, three mitochondrial (mt) and one nuclear (nu) data set Fig. . Completstresana . The thiw = 6. Within its scoring function gaps were treated like ambiguous characters on nucleotide level. On amino acid level we used the BLOSSUM62 substitution matrix. Positions identified by ALISCORE or suggested by GBLOCKS as randomly similar were removed and single genes were concatenated for each data set and each approach. Percentage of remaining positions after masking was plotted for each alignment and masking approach  trees were estimated with RAxML 7.0.0  and the The authors declare that they have no competing interests.PK and BM conceived the study, PK designed the setup and performed all analyses. PK and BM wrote the paper with comments and revisions from KM, BT, BMvR and JWW. Beta testing of ALISCORE and processing of ESTs was conducted by KM. Sequence data was provided by KM , JD (12S + 16S rRNA), and BMvR . All authors read and approved the final manuscript.Table S1. Detailed analyses results. Detailed results including lists of the percentage of remained positions after alignment masking per data set and alignment method. Given are all considered clades and corresponding bootstrap values (>50%) per data set, alignment method and (un)masked approach as well as all Neighbor-Net graphs.Click here for file


We have previously combined statistical alignment and phylogenetic footprinting to detect conserved functional elements without assuming a fixed alignment. Considering a probability-weighted distribution of alignments removes sensitivity to alignment errors, properly accommodates regions of alignment uncertainty, and increases the accuracy of functional element prediction. Our method utilized standard dynamic programming hidden markov model algorithms to analyze up to four sequences.BigFoot, for performing phylogenetic footprinting on greater numbers of sequences. We have developed a Markov chain Monte Carlo (MCMC) approach which samples both sequence alignments and locations of slowly evolving regions. We implement our method as an extension of the existing StatAlign software package and test it on well-annotated regions controlling the expression of the even-skipped gene in Drosophila and the α-globin gene in vertebrates. The results exhibit how adding additional sequences to the analysis has the potential to improve the accuracy of functional predictions, and demonstrate how BigFoot outperforms existing alignment-based phylogenetic footprinting techniques.We present a novel approach, implemented in the software package BigFoot extends a combined alignment and phylogenetic footprinting approach to analyze larger amounts of sequence data using MCMC. Our approach is robust to alignment error and uncertainty and can be applied to a variety of biological datasets. The source code and documentation are publicly available for download from  The identification of conserved DNA sequences by comparative genome sequence analysis has been widely used to annotate both protein-coding and gene regulatory elements in a wide variety of taxa -5. While"Statistical alignment"  methods Drosophila cis-regulatory modules demonstrate how removing the traditional dependence on a single alignment increases the accuracy of functional element predictions. The improvement was most prominent when there was alignment ambiguity in functional regions due to binding sites that were not highly conserved. While SAPF is used to discover new motifs, the MORPH software package modifies a simple probabilistic aligner to detect and align instances of known motifs that have been previously characterized as position state weight matrices [Statistical alignment models can be modified to simultaneously align sequences and detect functional elements. By doubling the number of states in a hidden markov model (HMM) aligner in order to model both quickly evolving  and slowly evolving  elements, we recently introduced SAPF , a software package which analyzes a probability-weighted distribution of alignments in order to identify sequence elements that are evolving at a reduced rate . Resultsmatrices . Here toWhile these studies present strong evidence for the benefits of using statistical alignment to detect regulatory elements, they are limited in the amount of sequence data they can analyze. Both SAPF and MORPH use standard HMM algorithms  to compuMarkov chain Monte Carlo sampling techniques ,26have bWhile traditional alignment algorithms assume identical mutation rates throughout the sequence, we introduce an alternative evolutionary model allowing for rate heterogeneity by modeling the evolution of both quickly and slowly evolving regions. In our model, a two-state HMM emits a sequence of conserved (slow evolution) and non-conserved (fast evolution) states at the root of the tree. This defines an alternating series of conserved and non-conserved segments, allowing our model to represent both neutral sequences expected to exhibit higher mutation rates and functional sequences undergoing purifying selection. Each segment evolves independently along a phylogenetic tree according to a pairwise alignment model which allows for insertions, deletions, and substitutions on each branch of the tree. While the StatAlign package jointly estimates both the tree and the alignment, we condition the analysis on a user-inputted phylogenetic tree in order to estimate the alignment and locations of quickly and slowly evolving regions more efficiently.Our model is a reformulation of the SAPF multiple HMM with two main differences. As in , we modeAnother minor difference is that the BigFoot model does not allow for insertions at the exact boundaries of functional regions. This modification was necessary to create unambigious and reversible MCMC proposals, and is a biologically relevant modification since an insertion event in one sequence should not define the beginning (or end) of a conserved region.Methods section and supplementary sections S1.1 and S1.2 [see Additional file Our alignment model is expressed as a pairwise HMM transducer ,30, a coWe base our transducer on the 1992 approach of Thorne, Kishino, and Felstenstein (TKF92) . TKF92 mWe place a pairwise transducer on each branch of the phylogenetic tree in order to model the evolution from each ancestor to each descendent. Transducer theory ,30 showsOne limitation of our approach is that our model does not allow for the creation or deletion of conserved regions along the tree. Since the annotation of fast or slow characters emitted by the root is conserved in the descendent sequences, the model cannot detect the loss or gain of binding sites. For this reason, when testing BigFoot we discarded sequences with long deletions from analysis.The full likelihood of a tree is equivalent to the full emission probability of the multiple-HMM. Unfortunately, this likelihood cannot be calculated quickly, as the time complexity of the Forward algorithm for a multiple HMM grows exponentially with the number of sequences. Instead of direct computation via dynamic programming, we apply a Bayesian MCMC method with data augmentation.All model parameter densities are estimated using MCMC sampling. Exponential priors with expectation 1 have been used as priors for insertion-deletion parameters in the alignment transducer and for all free parameters in substitution models provided with the software package. For parameters responsible for annotation , we allow the user to input either Beta or uniform priors on these parameters. This allows the user to tailor the analysis to their specific needs. For example, the user can set an informative prior to search for longer weakly conserved regions, or for very short and highly conserved regions. Alternatively, the user can set uninformative priors and allow the MCMC to estimate parameter distributions freely.The joint posterior distribution of alignments, trees and evolutionary parameters forms a high dimensional and complex distribution from which efficient direct sampling is most likely impossible. Therefore, we applied Markov chain Monte Carlo to converge to this prescribed distribution. After convergence, samples from the Markov chain provide correlated samples from the posterior distribution.extended alignments. This data contains information on how the observed sequences are aligned to ancestral sequences associated with internal nodes of the tree. We represent the unobserved ancestral sequences as a collection of gaps and Felsenstein wildcards in order to sum over all possible nucleotide values when calculating the total likelihood.The likelihood of a tree under the multiple HMM can only be easily calculated when we augment the tree with additional data known as Our MCMC walks on the joint distribution of the extended alignments, locations of fast and slow regions, and model parameters. The random walk comprises the following components:• Changing model parameters• Changing extended alignment• Shifting the boundary of an existing fast or slow region• Creating a new (or deleting an old) fast or slow regionMethods section. In each MCMC iteration, we apply a Metropolis-Hastings move to alter one of these components, and select moves with fixed probabilities that were chosen to enhance mixing.The first two types of moves are described in , and theThe Markov chain provides correlated samples from the posterior distribution of alignments, locations of fast and slow regions, and evolutionary parameters. To report posterior probabilities for phylogenetic footprinting predictions we take the approach of ,5,20, coMultiple sequence alignment samples can be summarized in several ways. Unlike other authors , we founcis-regulatory module in four Drosophila species: D. melanogaster, D. erecta, D. pseudoobscura, and D. willistoni. This 485 base pair region has been found to regulate the expression of the homeodomain encoding protein eve in the second stripe of the developing Drosophila embryonic blastoderm [D. melanogaster sequence [D. melanogaster or D. pseudoobscura. Four were biologically characterized as "weak-affinity" binding sites [D. melanogaster due to an absence of orthologous sequence in both closely and distantly related Drosophila species [As a first test of the accuracy of the MCMC results, we ran BigFoot on a relatively small dataset to compare the results with the exact dynamic programming predictions of SAPF. The two methods were expected to return similar, though not identical, results. This is because SAPF and BigFoot use slightly different alignment HMMs on each tree branch. We analyzed a astoderm . The REDsequence , and thesequence . Figure ng sites  which co species ,37.Drosophila 12 genome consortium has completed the sequencing of 12 Drosophila genome sequences exhibiting a large range of evolutionary distances [D. melanogaster and D. grimshawi is greater than that between any pair of mammals when generation time is taken into account [eve stripe 2 enhancer region using ten of the 12 genomes in this dataset. We chose to remove two species, D. mojavensis and D. virilis, as both sequences contained numerous long deletions and were thus too divergent to be informative. Sequence data for all species were obtained from a set of pre-computed whole-genome alignments [The istances . For exaignments .Figure • Higher sensitivity to verified binding sites. While Figure • Finer nucleotide resolution for TFBS start/stop positions. When analyzing a small number of species, it may be difficult to identify the boundaries between quickly and slowly evolving regions, especially in a region where TFBS may be grouped close together. The results shown in Figure Additionally, in a closely spaced group of functional elements (bases 440-482) separated only by a small number of neutral bases, the small dips in posterior probability correspond more closely to the neutral regions when more sequences are added to the analysis. While the agreement is not perfect, these results are consistent with previous findings showing an increase in nucleotide resolution as more species are analyzed .In order to quantify the predictive accuracy of our results, we calculated receiver operating characteristic (ROC) curves for both sets of BigFoot results. The area under the curve (AUC), which has a maximum value of 100%, is a summary statistic that accounts for both the sensitivity and the specificity of the predictions. A value of 50% implies that the predictions are no better than random guessing. The methodology used for creating the ROC curves is described in . The curα-globin (αMRE) [To demonstrate that BigFoot can be applied to sequence data from vertebrates, we analyzed a 256 base pair region previously identified as the major regulatory element of human n (αMRE) . This ren (αMRE) ,41.To analyze this region, we downloaded the multiz28way alignment of the region from the UCSC genome browser . This alWhen analyzing the 12 vertebrate species, BigFoot detects six of the seven known binding sites with posterior probabilities of greater than 95%. However, the seventh binding site, notated in Figure Drosophila and the αMRE datasets.One of the most widely used alignment-based phylogenetic footprinting tools is the phastCons program, used to create the conservation track in the University of California at Santa-Cruz (UCSC) genome browser ,24. The eve stripe 2 enhancer. The core of the TFBS is very well conserved and BigFoot predicts the site with high probability, but there are substitutions towards the edges and short indels in some species, and the multiz alignment incorrectly aligns the binding site. As a result, phastCons detects all nucleotides in the TFBS with less than 5% probability. The ROC curve for the phastCons predictions is displayed alongside the BigFoot ROC curves in Figure In the command-line version of phastCons, however, it is possible to set an option for the program to incorporate indel information . We set αMRE enhancer results in phastCons failing to annotate any nucleotide the weakly conserved region discussed in the previous section (bAlt) with greater than 5% probability. While we cannot know if this region is a functional TFBS without further experimental analysis, examining a distribution of alignments ensures that this region is not incorrectly passed over during the conservation analysis.A similar error in the multiz alignment of the Since phastCons analyzes only one single alignment, the computational time to analyze a single region, around 30 seconds on a 2 ghz macbook computer, was substantially less than the 12-16 hours needed to analyze a single region with BigFoot. As a result, phastCons can be used to compute functional element predictions for the entire genome, while BigFoot can only be used to analyze individual regions of interest. However, for users who have identified specific genomic regions to study in detail, the benefit of controlling for alignment error and uncertainty by using BigFoot may justify the additional computational time needed for the analysis.The algorithms have been implemented in Java 1.5, and are part of the BigFoot software package available at: BigFoot requires the user to input a set of homologous DNA sequences (in FASTA format) and an evolutionary tree (in newick format) describing the phylogenetic relationships between the inputted species. BigFoot can construct an initial alignment of the sequences, or if the user has a previously computed starting alignment in FASTA format, they can set it as the starting alignment in the Markov chain. The user can also place either Beta or uniform priors on parameters modeling the difference between fast and slow states.Our aim was to build a software package for an insertion-deletion model that can be coupled with an arbitrary substitution model. Therefore we would like to give users the option to implement their own substitution models. In the software help file, we describe how users can extend this class to create their own substitution models. We currently provide a large selection of eight nucleotide substitution models including the Jukes-Cantor model , the KimOur program provides random samples from a Markov chain whose stationary distribution is the joint Bayesian distribution of sequence alignments, locations of fast and slow regions, and model parameters. This high dimensional joint distribution can be analyzed in several ways, ranging from an analysis of the posterior distribution of a single rate parameter to an investigation of Markov chain convergence using a log-likelihood trace or a separate multidimensional autocorrelation analysis. We implemented a set of postprocess plugins which analyze data from the Markov chain and display in the graphical interface. In the software help file, we also describe how the user can implement their own postprocess plugin by extending the abstract class.We implemented the following plugins, each of which represents a tab in the graphical interface:Log-likelihood trace This plugin plots the log-likelihood trace and writes the log-likelihood values into a text file when the analysis is complete.• Current alignment This plugin shows the multiple sequence alignment in the current state of the Markov chain, along with the locations of slow and fast regions. Capital letters in the alignment represent slowly evolving regions in the current state of the chain, while lower case letters represent quickly evolving regions.• MPD alignment This plugin calculates and performs running updates of the Maximum Posterior Decoding alignment based on all previous samples of the Markov chain. Figure • Current tree This plugin graphically displays the tree inputted by the user.• 6-107 steps were required for convergence, depending on the number of sequences and their lengths. To remove the effects of autocorrelated samples, we took a sample of the chain after every 5000 iterations of the MCMC. For all examples, total computational time did not exceed 16 hours on a 2 ghz Macbook computer. Datasets larger than 12 species can also be analyzed, but may take longer to achieve convergence.We initially assessed convergence using a log-likelihood trace, and verified convergence using independent MCMC runs. We found that 10We have presented and tested an algorithm for co-sampling multiple sequence alignments, locations of quickly and slowly evolving regions, and a set of evolutionary parameters. Our likelihood engine evaluates an HMM transducer switching between fast and slow states, where the evolutionary models in the slow states indicate a reduced rate of mutation as a consequence of purifying selection. We also present a new MCMC transition kernel enabling the combination of sequence alignment and phylogenetic footprinting. We have demonstrated the accuracy of our method by comparing the results with a dynamic programming solution, and we have presented strong evidence for the convergence of our sampling distribution by running independent MCMC runs from different starting points and obtaining essentially identical results.eve stripe 2 gene in Drosophila. Our results exhibit two major potential benefits for analyzing additional sequences in comparative genomics approaches. Augmenting our dataset from four to ten sequences resulted in higher sensitivity towards experimentally verified binding sites and finer nucleotide resolution when detecting the exact boundaries for TFBS. The ability to analyze larger datasets was the primary motivation for extending the SAPF dynamic programming approach into BigFoot, and these results show that it is important for existing tools to have the capacity to analyze multiple sequence datasets.We tested BigFoot on the Drosophila) that were misaligned in the multiz alignments. While both these regions contained small indel events and multiple nucleotide degenerate sites, resulting in multiple plausible evolutionary histories, the posterior alignment distribution from BigFoot exhibited how our joint model reliably aligned all instances of the binding site together.By simultaneously estimating multiple sequence alignments and phylogenetic footprinting predictions, BigFoot correctly incorporates alignment uncertainty information into functional element predictions, and ensures that alignment error or ambiguity will not prevent the software from identifying slowly evolving regions. However, there is an additional benefit to our joint model. In some weakly conserved regions there is a highly conserved core of 3-5 bp, but conservation drops off slightly towards the edges of the site. In these cases, a simple aligner will often correctly align only the binding site core while misaligning the outer regions, or misalign the entire binding site. Our combined model, however, recognizes that a conserved core may indicate the presence of a slowly evolving region, and thus could be surrounded by other conserved nucleotides. In these cases, the model will push weakly conserved positions together to align the full binding site. As a result, BigFoot not only detects the binding site, but also increases the accuracy of the posterior alignment distribution. The two best examples of this phenomenon are the two conserved regions  sequence. Transitions to a slow state only occur when there is a "slow" character emitted in the ancestral sequence, and the same is true of fast states and fast characters. The model switches between fast and slow states only when a character type switch is emitted from the ancestral root. As previously discussed, the ancestral root node switches between emitting fast and slow characters according to a basic hidden markov model.The transducer model contains two "wait" states, W1 and W2, to which the model transitions while waiting for input from the ancestral sequence. These transitions occur after the transducer has finished processing the previous character from the ancestral sequence , and the transducer will remain in the wait state until the ancestral sequence emits the next character. This allows the transducer to pause while other evolutionary events, such as indels on a different half of the tree, occur in other sequences. A second wait state is needed since unlike the insertion state, the delete state cannot self-transition. This second wait state, which only the delete state can access, creates an effective self-transition that allows for the modeling of long deletion events. The full transition matrix for our HMM transducer is shown in supplementary section S1.2 [see Additional File In order to combine both alignment and phylogenetic footprinting, we needed to introduce new sampling moves for our random walk. At each state in the chain, we kept track of the points where the HMM transducer switched between fast and slow states, corresponding to a predictive switch between neutral sequences and functional elements. For display purposes  an existing boundary and calculate the effect on the overall likelihood. We fix the root sequence and switch a small number of nucleotides (determined by a geometric distribution) in the root between fast and slow. Figure • Creating a new pair of boundaries. This move proposes either a new slowly evolving region in a stretch of neutral sequence  or low conservation (for new quickly evolving regions).Deleting an existing pair of boundaries This move is the exact reverse of the move described above, and is also pictured in Figure • After each of these moves, we calculate the probability of proposing the new state and the probability of back-proposing the old state, along with the resulting change in likelihood. We then accept or reject the move, with an acceptance probability set by the Metropolis-Hastings ratio.• Project name: BigFoot• Project webpage: • Operating system: Platform independent• Programming language: Java• Other requirements: Java Virtual Machine 1.5 or higher• License: GNU GPLRS wrote the majority of the phylogenetic footprinting code and performed the biological dataset analysis. IM, AN, and RL wrote the alignment software and postprocessing code. All authors contributed to the manuscript. JH encouraged the research and supervised the project.Technical Methods. Additional technical information describing the BigFoot transducer parameters and the algorithm used to estimate MPD alignments on-the-fly.Click here for file