Amino acid sequences were evolved along the trees and input to the tested methods; from their calculated distances we infered trees whose topologies we compared to the reference trees.We have developed an alignment-free method that calculates phylogenetic distances using a maximum-likelihood approach for a model of sequence change on patterns that are discovered in unaligned sequences. To evaluate the phylogenetic accuracy of our method, and to conduct a comprehensive comparison of existing alignment-free methods (freely available as Python package We find our pattern-based method statistically superior to all other tested alignment-free methods. We also demonstrate the general advantage of alignment-free methods over an approach based on automated alignments when sequences violate the assumption of collinearity. Similarly, we compare methods on empirical data from an existing alignment benchmark set that we used to derive reference distances and trees. Our pattern-based approach yields distances that show a linear relationship to reference distances over a substantially longer range than other alignment-free methods. The pattern-based approach outperforms alignment-free methods and its phylogenetic accuracy is statistically indistinguishable from alignment-based distances. Tasks like database searching, sequence classification, phylogenetic tree reconstruction and detection of regulatory sequences are ubiquitous in bioinformatics. Most methods performing these tasks are based on (automated) alignments; however, alignment-free methods exist for solving the tasks. Recent years have seen an increasing number of alignment-free methods  Euclidean distance; dS, the standardized Euclidean distance; dF, a distance based on the fractional common k-mer count; dP, a distance based on probabilities of common k-mer counts under a multiplicative Poisson model; dC, the composition distance; dW, the W-metric; dLZ, a distance based on Lempel-Ziv complexity; dACS, a distance based on the Average Common Substring length. Methods dS and dP require a set of equilibrium frequencies of amino acids. We supply amino acid frequencies used in the derivation of the JTT model  of n (m) characters taken from the alphabet 𝒜 which contains c different characters {a1,…, ac}, eg all amino acids. Most alignment-free methods operate on words of length k, so-called k-mers: there are w = ck such different words. We represent the set of k-mers in X (or a derived property) by vector vX =  (similarly for k-mers in Y); the parameter k is always implied. Each vector element describes the abundance of k-mer i.We represent a biological sequence by a string X and Y is calculated using ciX(and ciY), the count of k-mer occurrences in X (and Y).The (squared) Euclidean distance was introduced into sequence comparison by dE yields values about twice the number of mismatch counts obtained from alignments.Later, dE without incurring the computational problems associated with the slightly better performing Mahalanobis distance (The standardized Euclidean distance was found to improve on distance .fiX (fiY), the relative frequencies of k-mer occurrences in X (Y), by their standard deviations siX (siY) as calculated from a set of equilibrium frequencies  denote the common k-mer count.Let F, the fraction of common k-mers between X and Y, ranges from 0 to 1 and dF transforms this into a distance: ɛ, a small value added to prevent taking the logarithm of zero  and proceed as follows. We accept p1 and discard all positions overlapping the corresponding k-mer. We repeat this process with the remaining positions until the list is exhausted; the accepted positions represent recurrences of i. FP refers to the Poisson probability distribution function and its parameter Ei is the expected count under a set of equilibrium frequencies. P(x ≥ CiXY) is the probability that we observe a k-mer count at least as high as that between X and Y.Here, in the calculation of k-mer occurrence of order k − 2 (assuming word length k ≥ 3) we predict the probability p0 of a word (the ci refer to its characters) from the probabilities p (ie relative frequencies) of appropriate shorter subwords. To obtain the expected count EX of a k-mer in X, we re-arrange Equation 7 slightly.The last word-based alignment-free method considered here is the composition distance of k-mer occurrence counts cX in X: vX = (cX − EX)/EX. Then we calculate the correlation between X and Y as the cosine of the angle between their composition vectors, and obtain a normalized dissimilarity dC.We can now assemble the composition vector  for k-mefiX − fiY, between all pairs of amino acids, are weighted by their entries Wij in scoring matrix W. Differences in amino acid composition, 1**): c(X) denotes the Lempel-Ziv complexity of X, and XY refers to the concatenation of X and Y.L = ∑i=1n ℓiXY/n, where ℓiXY is the length of the longest string starting at Xi that exactly matches a string starting at Yj. L provides a normalized length measure, from which we obtain an intermediate (asymmetric) distance d and finally dACS.Most recently, We use pattern discovery to find regions of similarity (presumed homology) occurring in two or more sequences; no alignment is necessary. To estimate phylogenetic distances, the patterns are considered to be local alignments. Adopting this point of view enables us to apply an established maximum-likelihood (ML) approach. Both the application of pattern discovery, and distance estimation by ML, represent novel steps in this context. We infer pairwise distances from the sequence data covered by patterns, yielding a distance matrix. Distance-based tree inference then proceeds by conventional means.k-mer that, by definition, occurs identically in these sequences.There is a fundamental distinction between our pattern-based approach and word-based methods: patterns are expressive enough to accommodate similar but non-identical stretches of sequence (see below), allowing us to calculate distances from the differences contained therein. Methods based on words cannot extract differences between two sequences from the sequence data covered by a P to be the regular expression 𝒜(𝒜 ∪ {‘.’})*𝒜, ie P starts and ends with amino acids, and may contain wildcard characters. A subpattern of P is any substring that is a pattern. Call P a < L, W > pattern (L ≤ W) if any subpattern of length ≥ W has ≥ L characters ∈ 𝒜. A pattern P has support K if it occurs (has instances) in K sequences. A pattern P can be made more specific by replacing wildcard characters by characters ∈ 𝒜 and/or extending P to the left or right. Call P maximal with respect to a sequence set S if making P more specific reduces its total support (irrespective of the number of sequences).We briefly introduce some basic terminology for TEIRESIAS; for more details, including a description of the algorithm, the reader is referred to L, W > patterns (with support ≥ K) in a set S of unaligned sequences. In the context of this work, we set L = 4, W = 16 and K = 2. As described below, distances between sequences are calculated using paired residues from patterns. Two residues are said to be paired if they occupy the same column, ie position in different instances of a pattern.We are now ready to state the behaviour of the algorithm: TEIRESIAS finds all maximal < dPB between two sequences Si and Sj is computed as follows: first, we filter out patterns with more than one instance in any sequence, for two reasons. a) This reduces the number of false positives, ie patterns with non-homologous instances that arise purely by chance. b) It also ensures that the distance from a sequence to itself is zero, a mathematically desirable property.  Second, all instances of patterns occurring simultaneously in (at least) Si and Sj are concatenated, resulting in two new sequences S′ij and S′ji of the same length. For example: pattern P1 = “A.RC” occurs in S1 and S2 with instances “AIRC” and “ALRC”, pattern P2 = “MA..D” occurs in S1, S2 and S3 with instances “MAMAD”, “MAVID” and “MALVD”. After concatenation, we obtain S′12 = “AIRCMAMAD” and S′21 = “ALRCMAVID”. Note that a pattern may occur in three or more sequences, in which case we project it on multiple pairs of sequences. For example: P2 contributes to ,  and . Also note that generally S′ij (and S′ji) will differ in length from S′ik (and S′ki) because the patterns occurring simultaneously in (at least) Si and Sj will differ in number (and number of residues they cover) from those appearing in (at least) Sk and Sj. For example: the length of S′12 (and S′21) is 9, whereas the length of S′13 = “MAMAD” (and S′31 = “MALVD”) is 5. Third, these new, concatenated sequences are used to estimate pairwise distances. This is done by applying a maximum-likelihood approach that optimizes with respect to a model of amino acid evolution. For the purpose of this work we use the JTT model , setting L = k leads to elementary patterns capturing a subset of all k-mers. The only difference is that K = 1 for k-mers (a k-mer may occur only once) whereas we use K = 2 for TEIRESIAS (a pattern must occur in at least two sequences). Thus we see that higher values for L reduce the number of false positives. For our distance calculation, however, we need patterns capable of accounting for differences between sequences, hence we require W > L. Our approach differs from word-based methods: only the use of patterns allows us to formulate and satisfy this requirement. In preliminary experiments on data described in Section 4.2, we tried several higher values for L with W > L first. We found for L = 4, W = 16 (a ratio of L/W = 0.25), the values that we use throughout Section 4, all pairwise distances are defined, ie every pair of sequences is covered by at least one pattern with an instance in both sequences. For W = 8, corresponding to a ratio of L/W = 0.5, and higher values of W, approaching the ratio L/W = 0.25, the number of undefined distances is 229, 127, 63, 32, 23, 8, 5, and 2 out of 8667. Our rationale for setting TEIRESIAS parameters to on phase . For theL, W > patterns. The ratio L/W determines the minimum similarity any subpattern must possess: it effectively specifies a local similarity threshold. Thus, undefined distances mean that no pair of substrings reach or exceed this threshold. Our solution to the problem is to make sequences more similar by encoding them in a reduced alphabet. Following L = 4, W = 8 are defined.Undefined distances point towards a problem: some sequence pairs are too divergent—no pair of substrings can be described by (elementary) < L, the phylogenetic accuracy is virtually independent of the particular choice of L, and largely depends on the ratio L/W  majority consensus and consistency. We discover patterns as before but introduce an intermediate step before distance estimation. We record paired positions across all patterns. For any two sequences Si and Sj, we accept positions  if and only if a) pik is paired with pjn more often than with any other position in Sj, b) pjl is paired with pim more often than with any other position in Si, and c) pik = pim and pjl = pjn, ie the positions in a) and b) correspond to each other. For example: assume p1,42 is paired with p2,52 three times and twice with p2,43. Hence, condition a) is met. Assume further that p2,52 is paired with p1,42 once, meeting condition b). Additionally, condition c) is met, thus we accept pair . This ensures that every residue participates at most once for a given sequence pair in the distance calculation step. For parameters L = 4, W = 16, the constraints prove to be stringent and discard most of the data.One property of TEIRESIAS is that each residue can  participate in multiple patterns. This may lead to situations where a particular residue in one sequence pairs with two or more different residues in a second sequence. For example: assume patterns We use a birth-death process to model cladogenesis  and sampbirth = 10.0 and death = 5.0, with extant ∈ corresponding to a sample fraction of . The induced pairwise phylogenetic reference distances have medians of  substitutions per site; their upper and lower quartiles are within 0.35 units of these values. For later use, we label the first, fourth and last set as having small, medium and large phylogenetic distances. Sequences were evolved along the branches of the deviated trees using SEQ-GEN V1.3.2  or the chemical equivalence classes (CE) from Section 3.3.Here, we look at the performance of word-based alignment-free methods as a function of the length of k = 4 performs best for methods dE, dS, dF and dP as judged by their ranksums based on phylogenetic accuracy over all seven reference sets. Second- and third-ranking word lengths for dE and dP are k = 5 and k = 3. For dF these lengths have tied ranks, and for dS this order is reversed. Method dC performs best for k = 3, with k = 4 (k = 5) ranking second (third).For AA sequences, word length k yield lower ranksums. Methods dE, dF and dP perform best with word length k = 5. Second- and third-ranking word lengths for dE and dP are k = 6 and k = 7, for dF this order is reversed. For dS, word lengths k = 5 and k = 6 rank equal best, followed by k = 4. Again, method dC shows a preference for lower values: it performs jointly best for k = 4 and k = 5, followed by k = 3.For CE sequences, slightly higher values for k, the absolute values of this difference may vary considerably. This leads to a number of curves with different shapes when plotting the accuracy for a particular method on the Y-axis with the X-axis showing values for k : thus the parameter space is uneven, more so than for other methods.Taken together, these results indicate that, depending on the phylogenetic distance of the sequences, the word length most appropriate for analysis of a particular data set may well be different from the one performing best over all sets tested here. In practice, intermediate word lengths  should work well on a large variety of data sets.k-mer-based methods, we select the best-performing word lengths on the two alphabets as described above. We compare these methods to dACS, dLZ and dW, and to two variants of the pattern-based approach: dPB and dPBMC. As a baseline, we include dML, the maximum-likelihood (ML) estimate of phylogenetic distances between the  sequences.Here, we conduct a comprehensive comparison of all methods: we show their phylogenetic accuracy and assign statistical significance to our findings. For FR = 709.6 . This is highly significant  at or beyond the α = 0.05 level. Significant differences are found between the following pairs  is significantly better than all alignment-free methods not based on patterns. The ML estimate based on the correct alignment, dML, is significantly better than all traditional alignment-free methods and the pattern-based method working on original AA sequences. Note that dML is not significantly better than the two best-performing pattern-based variants working on CE sequences.The test statistic of the Friedman test (corrected for tied ranks) is dW (ranksum: 8281.0): differences to nearly all other methods are significant. The lack of phylogenetic accuracy originates from being based on 1-mers. For comparison, dE with AA, k = 1 incorrectly reconstructs the following number of trees for the seven reference sets: 38, 30, 39, 53, 58, 66, 59. These numbers are quite similar to the ones in dS and dF. In the case of dP, however, they are 59, 56, 65, 75, 59, 71, 65. This is an artifact of the method for k = 1 (and to some extent for k = 2) and vanishes for higher values. Also apparent is the poor performance of both parameterizations of dLZ and dC, the Lempel-Ziv and composition distances, respectively, with ranksums between 7359.5 and 7635.0.By far the worst method tested here is the W-metric dClustalW, dDialign, dMUSCLE and dProbCons, respectively). Aligning sequences that contain shuffled domains is inappropriate and our setup corresponds to an undesirable situation where, for example, in an automated environment tests have failed to detect the presence of domain shuffling. Hence, distances are estimated from alignments where not all homologous residues can possibly be aligned, and it is likely that in fact a substantial fraction of aligned residues are non-homologous.We now describe our findings from the reference set with simulated domain shuffling data. We apply the same alignment-free methods with parameter settings as before on the unaligned, partly shuffled sequences. Additionally, we run a number of multiple sequence alignment (MSA) programs on these data , and estFR = 270.3, N = 100, k = 22) detects the presence of a difference that is highly significant  at or beyond the α = 0.05 level. However, pairwise differences are statistically significant only between dClustalW and all other methods; this is likely due to lack of statistical power. Two parameterizations (CE and AA) of the pattern-based method dPB, L = 4, W = 16, rank jointly first with ranksums of 1011.5: they reconstruct 11 out of 100 trees incorrectly. This is followed jointly by dPBMC, with CE, L = 4, W = 16, and dS and dF, both with CE, k = 5. Their ranksums are 1066.5, and reconstruct 16 incorrect trees. The number of incorrectly reconstructed trees for alignment-based approaches are as follows (ranksum in parentheses): dClustalW: 71 (1671.5), dMUSCLE: 28 (1198.5), dProbCons: 25 (1165.5), and dDialign:21 (1121.5). Three out of four alignment-based approaches are among the seven worst-ranking methods. Interestingly dDialign, the best-performing of these approaches, uses a local alignment strategy; it occupies rank eleven jointly with two other methods. Conversely, what we just described means that eg dLZ, working on AA sequences, one of the worst-performing alignment-free methods as tested here, has a higher phylogenetic accuracy than three out of four combinations of MSA program and ML estimate, and even dW is significantly better than dClustalW on these data. Overall, the results show that alignment-free methods may perform better than alignment-based approaches, especially on non-collinear sequence data, as alignment-free methods do not make assumptions of collinearity.The Friedman test  and Fitch-Margoliash FM:  referenck as in Here, we inspect the behaviour of pairwise phylogenetic distances. The BAliBASE alignments yield 8667 reference distances, of which < 2.5% have ≥ 5.0 substitutions per site (36 distances are ≥ 10.0). In what follows, we consider reference distances < 5.0, ie < 500 PAMs. Note that “distances of 250–300 PAM units are commonly considered as the maximum for reasonable distance estimation” . Figure Ideally, a method calculates distances that are linear to the reference distances, as this allows for a simple (linear) transformation of its results to obtain the actual values (which are of intrinsic interest). We find that no method is linear over the whole range considered. Therefore, we apply linear regression and report results where the linear range is substantial. This is somewhat subjective; however, we note that it serves as a descriptive tool to understand the accuracy of the methods.dPB with CE sequences, L = 4, W = 16, shows a linear relationship between reference and estimated distances for up to 2.0–2.5 substitutions per site. Linear regression for all points below 2.0 yields y = 0.0794 + 1.2698x with a correlation coefficient (CC) of 0.8015. Higher distances are increasingly underestimated as saturation comes into effect and limits most distances (> 99%) to values < 3.5.y = 0.0668 + 0.7790x, CC = 0.7970); saturation limits most distances (> 99%) to values < 2.5. For the variant dPBMC with CE sequences, L = 4, W = 16 . Again, most distances (> 99%) are limited to values < 3.5, however this curve shows less saturation and more scatter.When patterns are discovered using the original BAliBASE sequences  as oppos, W = 16 , the lindE does not yield values that can be interpreted in units of substitutions per site. Instead, they relate to mismatch counts . Most of the data (> 99%) have an Euclidean distance of < 1200; data for k = 5, with CE sequences are very similar and omitted here.The (squared) Euclidean distance h counts  and are dS has no single discernable linear relationship, with most data (> 99%) taking on numerical values of < 200  being < 4000, although the scatterplots look almost identical.Similarly, dF distances for up to about 0.75 substitutions per site , ie would be undefined without adding ɛ as no k-mer is common to a sequence pair. This problem occurs especially for short, divergent sequences in set 1 of BAliBASE. For k = 5, with CE sequences, this number drops down considerably to 15.1%, and the plot exhibits more scatter.We find a linear relationship between reference and per site . Linear dP . We find most distances > 99% are limited to values < 1.65. For CE sequences, most distances > 99% are limited to values < 1.1.The method per site . Linear dACS and dF. The linear relationship is a desirable property, and likely explains the higher phylogenetic accuracy of the pattern-based approach.Comparing the various distributions, we find that all three versions of our pattern-based approach yield pairwise distances that exhibit a linear relationship to phylogenetic reference distances for up to about 2.0–2.5 substitutions per site. This constitutes a considerable increase from a maximum of 0.75–1.0 substitutions per site for methods FR = 579.2  is highly significant  at or beyond the α = 0.05 level. Significant differences are found between the following pairs  are significantly better-performing than two pattern-based variants although not than dPB with CE, L = 4, W = 16 (ranksum: 1239.0). This version significantly outperforms all but four alignment-free methods not based on patterns. Again, both parameterizations of the composition distance and the W-metric trail behind, with ranksums of 1968.5, 2171.0 and 2338.0, respectively. Similarly to our previous analysis, most alignment-free methods are statistically indistinguishable. Ranksums for methods 8–18 range from 1583.0 to 1755.0, a difference of 172.0. On this data set, dPBMC is only marginally better (ranksum: 1570.0). A possible explanation is apparent from dPBMC performs poorly on reference set 7 (large phylogenetic distances) in comparison to both parameterizations of dPB. We find 1986 out of 8667 pairwise phylogenetic reference distances (ie 22.9%) in BAliBASE have ≥ 3.0 substitutions per site.The Friedman test statistic dPB to be statistically indistinguishable from this estimation, while most alignment-free methods rank significantly worse on ordinary, non-shuffled sequences. However, on non-collinear sequences we show that most alignment-free methods reconstruct trees more accurately than approaches based on automated alignments. In fact, these alignments should not be used as they largely align non-homologous residues. The inspection of ClustalW alignments reveals artifacts of this method: it forces most residues to align with other (non-homologous) residues, and places too few gaps.We present here for the first time a comprehensive evaluation of alignment-free methods with respect to their accuracy in reconstructing the phylogenetic relationship among a set of sequences. We show that the performance of most methods is statistically indistinguishable from another. The pattern-based approach as introduced by us here proved to be significantly better than most previously established methods. At the same time, we provide a point of reference for alignment-free methods by measuring the maximum-likelihood (ML) distance estimate based on reference and automated alignments. In our tests, we found the best-performing version of our pattern-based approach dPB ranks higher than the equally parameterized variant dPBMC, although not significantly. The latter variant intuitively seems to do more justice to the concept of homology; however, we cannot provide a satisfying explanation for its worse perfomance. All three versions of our pattern-based approach result in distances that show a linear relationship to phylogenetic reference distances over a substantially longer range than any other alignment-free method considered here.In all three experiments we found that dPB with CE yields results as good as dPB with AA, and often yields considerably increased phylogenetic accuracy. We also tested the other alignment-free methods on sequences encoded in this alphabet. For any given parameterization, CE always improves performance on set 7 .We also utilized a different alphabet for amino acid (AA) sequences based on chemical equivalences (CE). We found that s see eg . Also, odML with dPB, especially on reference sets 5 to 7, ie large phylogenetic distances. Quite likely this will be possible only if new alignment-free methods incorporate models of sequence change.Finally, based on the data in


Transposons are "jumping genes" that account for large quantities of repetitive content in genomes. They are known to affect transcriptional regulation in several different ways, and are implicated in many human diseases. Transposons are related to microRNAs and viruses, and many genes, pseudogenes, and gene promoters are derived from transposons or have origins in transposon-induced duplication. Modeling transposon-derived genomic content is difficult because they are poorly conserved. Profile hidden Markov models (profile HMMs), widely used for protein sequence family modeling, are rarely used for modeling DNA sequence families. The algorithm commonly used to estimate the parameters of profile HMMs, Baum-Welch, is prone to prematurely converge to local optima. The DNA domain is especially problematic for the Baum-Welch algorithm, since it has only four letters as opposed to the twenty residues of the amino acid alphabet.We demonstrate with a simulation study and with an application to modeling the MIR family of transposons that two recently introduced methods, Conditional Baum-Welch and Dynamic Model Surgery, achieve better estimates of the parameters of profile HMMs across a range of conditions.We argue that these new algorithms expand the range of potential applications of profile HMMs to many important DNA sequence family modeling problems, including that of searching for and modeling the virus-like transposons that are found in all known genomes. These "jumping genes" share features and origins with viruses, though they differ from viruses in that they are usually unable to leave one cell to affect another [Transposable elements  , the popThe process of creating a profile or a consensus sequence from a set of sequences known to belong to a sequence family begins with the creation of a multiple alignment via insertions of gaps into each sequence so that the residues of the sequences line up. A score matrix is used to evaluate the possible ways of creating these alignments, with preference given to exact matches or chemically feasible mismatches, and to contiguous gaps. To create a consensus or profile, each column of the multiple alignment is first examined for the number or fraction of gaps, to determine if the column should be considered part of the profile (or consensus), rather than an insertion column. All of the non-insertion columns are tabulated for their content of each residue, for instance the number of adenine ("A"), cytosine ("C"), guanine ("G"), and thymine ("T") nucleotide bases. When constructing a profile, the relative frequencies of residues in a column are used to create a score matrix reflecting the column's divergence and bias .The reliance on a single multiple alignment is problematic, especially since transposon sequences can be very poorly conserved. There are several popular multiple alignment programs, including ClustalW , T-Coffedτ at time τ ∈ 1..K depends on an unobserved (hidden) state hτ. The general form of an HMM can be written asUse of profile hidden Markov models (profile HMMs) ,28, probeτ and tτ are probability distributions, and the hτ form a Markov chain. In this article we will follow the nomenclature of [eτ (·|hτ) as the "emission" distribution and to tτ (·|hτ -1) as the "transition" distribution. Figure where ature of  and refetτ ·|hτ - as the "τ associated with exactly one residue dτ. The unobserved states of the Markov chain represent the unknown correspondence between the observed sequence's residues and the positions of the ancestral sequence from which the observed sequence has derived by a process of mutation, deletion, and insertion. The hidden data can be viewed as the alignment between the sequence and the model. The states of the Markov chain in a profile HMM correspond to ancestral positions (and to deletions of them and insertions between them).One prominent instance of HMMs in sequence analysis is the profile HMM. Profile HMMs model the residues of a genomic sequence as the observed data Figure An important feature of the HMM is the existence of a tractable algorithm, Baum-Welch, for finding maximum-likelihood and maximum a-posteriori values for its parameters . The BauWe recently introduced two variants of the Baum-Welch (BW) algorithm : ConditiCBW  is an alIn the context of the profile HMM, the CBW algorithm is no less computationally efficient than the BW algorithm . This efficiency is possible because the algorithm uses the same dynamic programming recursions as Baum-Welch, and most of these values do not need to be recomputed when the parameters affecting only one model position are altered.Even with the improvements garnered by conditional maximization, the CBW algorithm often converges to local optima that are far from the global optimum. Through exploratory analysis using simulated data and profile-profile alignments, we have found that these locally optimal profile models often align with the correct profile (from which the data were simulated) after local shifts of the model positions.We thus designed the Dynamic Model Surgery (DMS)  algorithPreviously we found that under the controlled conditions of simulation studies, the new algorithms outperform Baum-Welch alone, with the best results achieved when CBW and DMS are used together . In thosIn , we demoOur goal for the simulation study was to evaluate the relative benefits of each algorithm across a range of conservation levels. We randomly generated profile HMMs for each of the seven conservation levels in the range from .3 to .9. From each of these "true" profile HMM models we drew a set of sequences; by design the set of sequences drawn from a profile with conservation level .5 are about 50% conserved . We then used these training sets to estimate the parameters of a profile HMM, and compared these estimated parameters to the parameters of the "true" model. The results we provide here compare the log-likelihood of the training sequences across each combination of algorithms . We also generated another set of sequences from the same "true" profiles and used these for cross-validation. Since the algorithms are deterministic but depend on the starting values of the parameters, we used the same random "starting" profiles for every algorithm.Figure The simulation study was designed to provide a simple setting in which to compare the methods. It did not capture the full complexity of real biological sequences. Further simulations could be performed to better reflect the natural biases in sequence residues, the transition/transversion biases in DNA substitution matrices, and other features of genomic sequences that might complicate the profile HMM parameter estimation process. Our intention is to continue exploring both simulated and real application environments for profile HMMs.Our goal for the transposon study was to determine if the same pattern of improvement is seen for profile HMM models of sequences generated by natural biological processes as that seen with data simulated from profile HMMs. Real genomic sequences are generated by a complex set of biochemical processes that are never fully representable in a mathematical model, and it is possible that our simulation results fail to capture an important feature of real genomic data.We aim to build profile HMM models to identify transopon elements in the mouse and human genomes. Due to the shorter generation time in mice, transposons are on average 1.7 times more diverged in the mouse genome than in the human genome, and are thus more difficult to identify [The mammalian interspersed repeat (MIR) family, at between 60 and 70 percent conserved, is one of the most diverged transposable element families identifiable in the human genome by current techniques. We put together a training set of about 200 sequences of MIR elements found in the human genome by RepeatMasker. We used these sequences to train profile HMMs  from a variety of random start profiles, and then evaluated the methods by searching for new elements in target regions of the mouse and human genomes known to be mutually homologous. We restricted the size of the training set to more easily compare the methods, though in practice we will use all available data when building models for deployment.In addition to calculating the number of elements found by each profile HMM training procedure that are homologous to known elements on the other genome, we also evaluated the procedures using a measure of the quality of the alignments. In particular we are interested in the quality of what we deem the "transitive alignments," the alignments between the known elements on one genome and the newly found elements on the other genome. We measured quality by the number of exact matches in each transitive alignment, plus the number of transitions  and pyrimidine  nucleotide sets, as opposed to transversions, which are the other, less probable mutations).To illustrate how the transitive alignment metric works, consider the human genome MIR element at hg18 chr10:364, 241-364,409. Figure The region of the mouse genome corresponding to the MIR element, according to the UCSC genome-genome alignment, is the complement strand of hg18 chr13:9,634, 994-9, 635, 139. One side of the RM alignment extends beyond the genome-genome alignment. The multiple alignment in Figure From this transitive pileup multiple alignment, pairwise alignments between the new "hit" (the putative new element) on the mouse genome and the known element on the human genome are implied (by removing the genome lines). These are what we refer to as "transitive alignments." The transitive alignment between the new element and the human MIR element is depicted in Figure Transitive alignments contain many more gaps than do non-transitive alignments. The majority of the gaps are flanking gaps; internally, these alignments are quite good. We have found that the boundaries between the flanking gaps and the internal alignments are usually defined by dot-gaps, indicating that there is a gap in one of the genomes relative to the other. These genome alterations can force one hit to terminate while the other continues.Our comparison metric is the number of matches in the transitive alignments between the hits we find on one genome and the RM hits on the other genome, plus the number of transitions. For instance in the transitive alignment depicted in Figure Figure All three new algorithm variants show significant improvements over Baum-Welch. The same pattern of improvement is seen as in the simulation study: CBW with DMS is the best combination, CBW alone provides an improvement over BW alone, and the BW & DMS combination provides more improvement over BW than does CBW alone. In this case the differences among the three new combinations is only slight, and is within the margin of error.The high variability of the Baum-Welch results reflects the higher tendency of that algorithm to get stuck in local optima. The DMS algorithm was more difficult to apply to this context than to the simulated data because these data show many more deletions than insertions. To accommodate this, we had to adjust a tuning parameter that governs the threshold at which positions are inserted or removed. While we put some effort into finding the parameter value that optimized the model length, the resulting models had higher length variability than models in our simulation study. The HMMer and SAM packages required tuning of analogous parameters. Details are provided in the Methods section.The comparison to RepeatMasker and to existing profile HMM packages indicates the promise of the new approaches. The MUSCLE-HMMer combination seems to improve upon RepeatMasker, but the number and quality of transitive alignments for all of the methods in Figure Profile HMMs are a model-based alternative to the ad-hoc multiple alignment approaches common for sequence family modeling. Since different multiple alignment software packages can yield very different alignments, use of the profile HMM approach is more robust than relying on a single multiple alignment to construct a consensus or a position-specific score matrix.While profile HMMs are widely used for protein family modeling, and databases of profile HMMs exist for protein sequence families, including the popular Pfam  and SUPEThe simulation study and transposon study both demonstrate that the Conditional Baum-Welch algorithm is an improvement over the Baum-Welch algorithm, as is the Dynamic Model Surgery algorithm. The simulation study demonstrated a greater improvement from the DMS algorithm than from the CBW algorithm, with the best improvement when the algorithms are used together. In the transposon study, each of the new algorithm combinations provide a large improvement over BW alone, but we found little discernible difference among the new approaches. The DMS algorithm always converged on a model length near the RepeatMasker consensus length, though it required tuning of a threshold parameter to do so. The HMMer and SAM packages also require parameters to be tuned to control the model length, and these packages performed relatively poorly even after tuning.We have argued that improving transposon modeling is a worthwhile endeavor, and have shown that, by one metric, the new methods do improve upon the transposon-finding abilities of existing profile HMM parameterization methods. The new algorithms overcome one of the major impediments to the application of profile HMMs in the transposon modeling problem: that of poor parameter estimation. Already new repeat elements have been identified using these new techniques, and we will work with the transposon research community to develop the methods further.ατ (hτ) = ℙ, where the notation dτ denotes the first τ elements (e.g. residues) of observed sequence Θ are the parameters of the HMM.BW  iterativThe probability of the sequence K = |where ατ as a function of ατ-1 follows from the conditional independence assumptions of the Markov chain underlying the HMM:The dynamic programming recursion for computing βτ(hτ) = ℙ is the conditional distribution of the remaining K - τ components of dτ), which we denote by dτ-, given the state hτ at time τ. The recursion for calculating the backward value is similar to that of the forward value, only in reverse:The "backward" value hτ given the observed sequence Alone, either the forward or the backward values can be used to efficiently calculate the likelihood function (Equation 3). Together, the forward and backward values yield the conditional distribution of the hidden state τth element of sequence hτ at time τ. More complex priors are also possible via the usual procedure, with conjugate (e.g. Dirichlet) priors preferable for computational simplicity.With uninformative (e.g. Laplace) priors on the parameters, this can be interpreted as the posterior probability that the HMM emitted the e and the parameters of the "transition" distributions t such that each parameter value is proportional to the average  of the expected number of uses of the corresponding emission (or transition). For example, the update to the probability of emitting datum d at state h ∈ In each iteration, the BW procedure updates the parameters of the "emission" distributions 1(·) is the indicator function and where dτ is the τth residue in observed sequence where hA to another state hB isSimilarly, the BW update for the probability of transitioning from some state The BW algorithm iterates between calculating the forward-backward values and updating the parameters, until convergence. The convergence point depends deterministically on the data, and unfortunately also on the starting parameters. In the case of profile HMMs, the convergence point is highly sensitive to the starting parameters.The CBW algorithm updates subsets of the parameters as a group, conditional on the other parameters. Applied to the profile HMM, for example, the CBW algorithm separately updates the transition parameters and Match emission parameters associated with each model position  of transitioning from state i to state j is zero unless j ≥ i and (j - i) ≤ k for some fixed small constant k. For the Plan 7 and Plan 9 profile HMM models, the condition is satisfied with k = 5 (from a Match state at one position to the Deletion state at the subsequent position).For CBW to be as efficient as BW, the transition probability matrix of the underlying Markov chain must be relatively sparse and relatively diagonal. In particular, there must exist an ordering of the states ατ(hτ) for time τ as a function of the forward values for all states at the previous time (as in Equation 4), the values can be computed for state s as a function of the k previous states at the previous time, sinceWhen this condition is satisfied, the forward and backward dynamic programming recursions can proceed state-by-state rather than time-by-time. That is, instead of computing the forward values The CBW algorithm computes the forward and backward values row-by-row (state-by-state) rather than column-by-column (time-by-time). Since the calculation of the forward values in a row depends only on the parameters affecting the states of that row, and on the forward values of the previous row, the CBW algorithm only needs to recompute the forward values of the affected row. Since each forward value needs to be calculated only once, the total computational cost of the CBW algorithm is on the same order as that of the BW algorithm .DMS makes use of the same row-by-row parameter update computations that are used by CBW. While CBW uses these calculations to update the parameter values of a profile HMM, DMS uses them to dynamically change the model's structure. The DMS algorithm can also be used in conjunction with the BW algorithm, so long as BW is implemented as a "delayed" CBW, as described in .The DMS algorithm identifies positions at which there are an excessive number of insertions or deletions. Positions that are underutilized are removed, while positions at which there is a high occurrence of insertions are duplicated. The effect is dramatic: after convergence the profiles have a much higher log-likelihood than they do without the misalignment correction. We have shown in  that undAltering the model structure helps the parameter estimation algorithms escape from local optima caused by poor use of the model's positions. To understand this it is helpful to think of the positions of the profile HMM as corresponding to columns of a multiple alignment. The problem is analogous to having some of the alignment columns contain mostly gaps: these should be identified as "insertion columns"; in the profile HMM context, there should not be a match state associated with such columns (there should be an insertion state instead). Another problem is model positions that are overutilized: they represent more than one column of the multiple alignment, and since each position has only one match state, all emitted residues after the first are misrepresented as  insertions. This misalignment problem can be corrected by adding and removing model positions.DMS is inspired by the Model Surgery technique introduced in  and implξ. If more than some fraction ν (e.g. 50%) of the sequences have expected insertion counts exceeding ξ at a position j, then an additional position is added to the model after position j. If more than 1 - ν of the sequences have expected deletion counts exceeding ξ, then the position is removed from the model.The DMS algorithm uses the BW or CBW update calculations to determine if a model position is underused  or if a model position is overused (in which case a new position is introduced into the model). In particular, the expected counts for transitions into the Insertion and Deletion states are compared to a threshold j, the fraction j exceeds the threshold ξ, and the fraction ξ. The DMS algorithm requires computing, for each position The efficiency of the row-orientated update procedure makes it possible to incorporate structural changes immediately. The row-wise orientation of the CBW forward-backward procedure ensures that the dynamic programming calculations do not need to be recomputed except at the affected states. Backward values for subsequent rows are not affected by the change, since the backward recursion concerns only the subsequent states. Likewise, forward values for preceding rows are not affected, since they depend only on the states preceding them, which are unchanged.For the simulation study, we generated sets of sequences from profile HMMs with known parameters. We split each set of generated sequences into a training set and a test set, and then assessed how well the algorithms performed on the test set after being parameterized with the training set. The profile HMMs from which the training and test sequence sets were drawn were generated by first drawing one random 100-length "true consensus" sequence for each. We then set the multinomial emission distribution at each position of the "true" profile to assign more probability to that true consensus residue than to the others. For instance if the randomly drawn true consensus residue at a position was "A" and the conservation level was .5, then we assigned .5 probability to "A" at that position. For simplicity, we evenly divided the remaining (1.0 - .5) probability among the other residues. Each profile was then used to generate 100 training sequences and 100 test sequences. We also created four uniformly-distributed starting profiles for each true profile. We separately ran each algorithm on the 100 training sequences from every starting position.ν set to .5). The DMS threshold ξ was set to begin at .01 and to increase by ϵ = .005 whenever a cycle was detected. The algorithms were trained until convergence, with convergence defined as the average euclidean distance of all free parameters being less than 10-5. For additional details, as well as a discussion of the limitations of the simulation study, we refer the interested reader to [The algorithms employing Dynamic Model Surgery were set to insert a profile position whenever the number of sequences with insertion fractions exceeding the DMS threshold was at least 50% of all training sequences, and to delete a position whenever the number of sequences with deletion fractions exceeding the DMS threshold was at least 50% of all training sequences  and the MIR hits found by RepeatMasker in the target region. We then transitively aligned mouse profile HMM hits to human RM hits and vice-versa. For each transitive alignment, we computed the number of matches and transitions that are shown in Figure We now provide specifics of how the transitive transposon results were generated. The profile HMMs were trained using the 206 MIR elements found by RepeatMasker in human (hg18) chromosome 22 that are least 150 residues in length. The region on the human genome used to test the trained profiles was hg18 chr10:1-1,000,000. According to the chain/net genome-genome alignments from UCSC, the region has 550 fragments aligning to mm9 chr13:9,000,000-10,000,000. For this reason we used mm9 chr13:9,000,000-10,000,000 as the mouse genome test region. RepeatMasker identifies 15 MIR elements in the human genome that overlap with these chain/net fragments, and 13 MIR elements in the mouse genome that overlap with these fragments. 8 of these MIRs, 4 on each genome, transitively align via the fragments.The profile HMMs were trained from 18 different starting profiles per method. The profiles were of length 262 (the length of the RepeatMasker MIR consensus sequence) for the BW and CBW methods. For the methods employing DMS, the length of the starting profiles was set to the median length of the training sequences, which was 173. The Match emission distributions of these profiles were randomly drawn from Laplace distributions. The Insertion emission distributions were even (.25 on each residue). The Insertion and Deletion state transition probabilities were even . The profile HMM algorithms were all set to "local" mode, meaning that flanking insertions were disallowed and extra deletion-in and deletion-out transitions were possible. The transitions from the Match state were set with probabilities .9 to enter the DeletionOut state, .095 to transition to the subsequent Match state, .0025 to transition to the Insertion state, and .0025 to transition to the subsequent Deletion state. The transitions from the Begin state were set with probabilities .9 to enter the DeletionIn state, .09 to enter the first Match state, and .01 to enter the first Deletion state. The transition probabilities from the DeletionIn and DeletionOut states were set such that continuing the deletion had probability .9999, and ending it had probability .0001. All of these parameters were subject to alteration by the algorithms, except for the flanking insertion probabilities, which were fixed to disallow flanking insertions.The average length of the profile HMMs that were trained using DMS was 259.5 for DMS with BW, with a standard deviation of .9, and 259.1 for DMS with CBW, with a standard deviation of 1.0. The RepeatMasker consensus sequence for this transposon has length 262. The lengths after convergence are strongly influenced by a parameter, essentially equivalent to the "gapmax" parameter of HMMer or the "mainline_cutoff" parameter of SAM. We tuned this parameter to get model lengths reasonably close to the consensus length. As described below, we did the same for the HMMer and SAM models. Note that with these sequences the DMS algorithm exhibits some variation in the model length . The DMS length results are very resilient to starting profile length, with no notable difference between the length distribution of models starting from length 262 (not shown) and that of models starting from the median length of the training sequences (as shown here).-5 was enforced for all parameter values being trained, to prevent the algorithms from getting trapped with zero-valued parameters.Weak Dirichlet priors were used in all algorithms for all of the parameters. In the case of the Match emission distributions, the priors were set to Laplace distributions. Priors for the other parameters were set with the same parameters as the starting values described above, with the Match, Insertion, Deletion, DeletionIn and DeletionOut state transition priors scaled by the initial profile length, to reflect their multiple use. In addition to the priors, a minimum value of 10The algorithms employing dynamic model surgery (DMS) were set to insert a profile position whenever the number of sequences with insertion fractions exceeding the DMS threshold was at least 10% of all training sequences, and to delete a position whenever the number of sequences with deletion fractions exceeding the DMS threshold was at least 90% of all training sequences. This roughly reflects the heuristic used by the authors of RepeatMasker, who include a residue in a transposon's consensus sequence whenever the multiple alignment column contains at least 4 non-gaps. The DMS threshold was set to begin at .1 and to increase whenever a cycle was detected to the minimum amount that would change the fraction of sequences exceeding it.-7.The algorithms were trained until convergence, with convergence defined as the average euclidean distance of all free parameters being less than 10After training, the resulting profile HMMs were converted to HMMer-2 format. This conversion procedure effectively prepares the HMMer-style HMM for "FS" style search . We used the profile HMM's insertion distribution for the null model emission distribution, and .25 for the null model termination probability and for the loop-state (inter-hit insertion) termination probability. These values imply an expectation of only 4 insertions between hits, which is lower than we actually expect. All other model probabilities are transferred unchanged into HMMer-2's HMM format.The HMMer-2 hmmsearch program was used to search each of these converted HMM files against the relevant test region . No arguments were given to this program except for the HMM file and the sequence FastA file. The resulting hits that exceeded the predetermined threshold were then converted to cross-match-style pairwise alignments, then transitively aligned to the RepeatMasker hits on the other genome. The transitive alignments were counted and the matching residues tallied.For comparison we also ran the popular profile HMM software packages HMMer and SAM on the same training data sets, and computed transitive alignments exactly as described above. The documentation for HMMer-2 warns against its use for DNA sequence family modeling, and indeed the default options created tiny (length 1) profiles. Regardless of the options, HMMer-2 does not actually implement Baum-Welch, and so it requires pre-aligned input sequences. We ran ClustalW  using thThe SAM program no longer supports DNA sequence family modeling through the web interface, but version 3.5 of the SAM buildmodel program allows it with the "-alphabet DNA" option. We also supplied the "-mainline_cutoff" option, which controls SAM's variant of model surgery and thus effectively controls the profile length. We searched for the optimal value of this option separately for each profile that we built with SAM, among multiples of .05. Since SAM accepts both aligned and unaligned sequence inputs, we build a total of three SAM models: one from the unaligned sequences, and one each from the ClustalW and MUSCLE alignments used for the HMMer profile building process. The resulting profile lengths were 249 for MUSCLE alone, 240 with ClustalW, and 255 with MUSCLE. We converted the resulting SAM profiles to HMMer-2 format and calculated transitive alignments, as described above.The authors declare that they have no competing interests.PE developed the methods, programmed the software, carried out the simulation and transposon studies, and drafted the manuscript. JL contributed to the development of the methods and helped to draft the manuscript.


Sequence alignment is one of the most important techniques to analyze biological systems. It is also true that the alignment is not complete and we have to develop it to look for more accurate method. In particular, an alignment for homologous sequences with low sequence similarity is not in satisfactory level. Usual methods for aligning protein sequences in recent years use a measure empirically determined. As an example, a measure is usually defined by a combination of two quantities (1) and (2) below: (1) the sum of substitutions between two residue segments, (2) the sum of gap penalties in insertion/deletion region. Such a measure is determined on the assumption that there is no an intersite correlation on the sequences. In this paper, we improve the alignment by taking the correlation of consecutive residues.We introduced a new method of alignment, called MTRAP by introducing a metric defined on compound systems of two sequences. In the benchmark tests by PREFAB 4.0 and HOMSTRAD, our pairwise alignment method gives higher accuracy than other methods such as ClustalW2, TCoffee, MAFFT. Especially for the sequences with sequence identity less than 15%, our method improves the alignment accuracy significantly. Moreover, we also showed that our algorithm works well together with a consistency-based progressive multiple alignment by modifying the TCoffee to use our measure.We indicated that our method leads to a significant increase in alignment accuracy compared with other methods. Our improvement is especially clear in low identity range of sequences. The source code is available at our web page, whose address is found in the section "Availability and requirements". Under a rapid increase of genome data, the need for accurate sequence alignment algorithms has become more and more important, and several methods have been developed. Sequence alignment algorithm is designed for mainly two purposes. One purpose is to design for comparing a query sequence with the database which contains preobtained sequences, and another is to design for generating multiple sequence alignment. FASTA  and BLASIn addition to for database search, sequence alignment is used for generating multiple alignment. In the multiple alignment, the accuracy is more important than the quickness. The recent popular multiple alignment methods, such as ClustalW , DIALIGNAccording to Anfinsen's dogma  , for a sIn this paper, we propose another approach introducing a new metric defined on compound systems of two sequences. Most of alignments are based on finding a path that gives the minimum value to the sum of difference  for each residue pair between two sequences. Our method is to change the way defining the difference  above by computing this sum of differences by introducing a quantity through the transition probability between consecutive pairs of residues. The comparison of our method with the method of Lu and Sze gives the following: In the very difficult range that the sequence identity is less than 15%, our method improves the accuracy nearly 8% up, but the Lu and Sze method improves it nearly 1% up.First, let us establish some notations. Let Ω be the set of all amino acids, and Ω* be the Ω with the indel (gap) "*": Ω* ≡ Ω∪{*}. Let [Ω*] be the set of all sequences of the elements in Ω*. We call an element of Ω a residue and an element of Ω* a symbol. In addition, let Γ ≡ Ω × Ω be the direct product of two Ωs and Γ* ≡ Ω* × Ω*.A = a1a2 ... an and B = b1b2 ... bn, both of length n, where ai, bj ∊ Ω*. We also denote the sequences by u1u2 ... un, where ui =  ∊ Γ*, and we call ui a site in the following discussion. In general, the relative likelihood that the sequences are related as opposed to being unrelated is known as the "odds ratio":Consider two arranged sequences, p (a) is the occurrence probability of the given segment and p  is the joint probability that the two segments occur. In order to arrive at an additive scoring system, Equation (1) is typically simplified by assuming that the substitutions are independent of the location and there is no intersite correlations; namely, p (A) = ∏ p (a), p (B) = ∏ p (b) and p  = ∏ p . Thus the logarithm of Equation (1), known as the log-odds ratio, is now a sum of independent parts:Here, Wherea, b) occurring as an aligned pair to that occurring as an unaligned pair. The s  is called a score and S = ) is called a substitution matrix. These quantities (Equation (2) and (3)) are used to define a measure for pairwise sequence alignment [A and B.is the log likelihood ratio of the symbol pair  = f ) for a, b Ω. This a, b) is a normalized expression of the score s . By using this quantity, we define a normalized substitution matrix as M = ). Then a difference of A and B is defined byLet put A is equal to B the difference dsub  has a minimum value 0.When the sequence A and B) was the induction of the occurring probability. We could take more informative approach by including the intersite correlations. Crooks et al. tried one of such an approach [A and B. Under an assumption that each site of the sequences has Markov property, we propose a new measure for two sequences by adding a transition effect and its weight ε (a degree of mixture):One of the essential assumption for the above approach  called "Transition Quantity", in order to simplify the equation. Let ui, ui+1) be a normalized transition defined asHere we introduce a normalized transition ft  is a normalizing function:where A and B representing the "intersite transition" is defined asBy using the above quantity, a difference of dsub and dtrans:Consequently, we define a difference measure for two sequences by combination of two differences N times in the dataset, its weight is N-1/2. We estimated the transition quantity from the weighted frequencies of observed transitions as follows.Let us discuss how to estimate the transition quantity on 1.63) . This seon 1.63) . For a lon 1.63) . If a seL, N be the set of N sequences with length L:Let aik, ajk) be in the finite set Γ* and it is a symbol pair in the kth site of the L, N. In addition, let L, N , and let NA be the frequency of the sequence A in the set Let R be a mapping which represents the weighted frequency appearing the symbols  in L, N such thatLet p (v\u) be a transition probability from the symbol pair u to the pair v on Let u, v in the finite set Γ*.for u,v)) called "Transition matrix" by the elements u, v) asWe define a matrix T =  = log p (v\u) and ft is the normalizing function defined by the equation (12).where Measure based on TRAnsition Probability) is an alignment algorithm by minimizing the value of a certain objective function based on the transition quantity , i = 1, ..., m, j = 1, ..., n as in Figure where P1 =  and a final point PN =  if the following conditions are met:a "route" with an initial point αR, βR be maps from a route ℛ = {P1, P2, ..., PN} to a set Ω* such thatLet μR be a map from the route ℛ to the set of all symbol pairs Γ*(≡ Ω* × Ω*) such thatand A* and B* the alignment of A and B by the route ℛ:We call the following R(P) be the set of all routes with the final point P, that is,Let - ≡ Ω* × Ω, (2) Γ;-* ≡ Ω × Ω*, (3) Γg-≡ {*} × Ω, (4) Γg -≡ Ω × {*}, (5) wopen is a constant called gap "opening" cost; 0 ≤ wopen ≤ 1, (6) wextend is a constant called gap "extending" cost; 0 ≤ wextend ≤ wopen and (7) ε is a weight, 0 ≤ ε ≤ 1 . The difference between A and B by a route ℛ is given byLet us fix the following notations for the following discussion: (1) Γ*ds is a function from Γ* × Γ* to R such thatwhere A and B with respect to a final point P can be defined asThe degree of difference between A and B isHence the degree of difference between DAB by a dynamic programming technique as below. For a final point Pk =  and a route ℛ = {P1, ..., Pk} ∊ R(Pk), we haveWe calculate Q1 = , Q2 =  and Q3 = . Thereforewhere withl = 1, 2, 3. Thus we obtainfor wherel = 1, 2, 3.for Ql has three points Ql one step after. These points are precisely written asEach point Q1 = , Q2 = , Q3 = .when Dl ) can be obtained from one step before by the following recursion relations:The distances l = 1, 2, 3. The values Dl of initial point and those of the edge points are assumed asfor Moreover for other special cases, the recursive relation of the edge points satisfiesR1 (Pk), i = 1, ..., m,for ℛ ∊ R3 (Pk), j = 1, ..., n.for ℛ ∊ mn steps.This calculation is completed in In order to discuss the effect of using MTRAP algorithm in the iteration step of progressive multiple alignment, we modified the TCoffee , a consi1. Generating a primary library by using MTRAP2. Extending the library 3. Making a guide tree for the progressive step4. Constructing a multiple alignment by progressive strategyThe modified TCoffee uses the extended library obtained by the MTRAP algorithm for aligning.We compared the performance of MTRAP to those of the most often used nine methods: Needle, ClustalW2, MAFFT, TCoffee, DIALIGN, MUSCLE, Probcons, Probalign and TCoffee-Lu/Sze. The details of these nine methods are: (1) Needle, a global pairwise alignment using Needleman-Wunsch algorithm  containeDIALIGN ver. 2.2 , a methoTo measure the accuracy of each method, we used three different databases: HOMSTRAD  ,28, PREFwopen, wextend, ϵ. Consequently, MTRAP uses the followings for parameter values: wopen = fs (-11), wextend = fs(-0.3) and ε = 0.775.In order to avoid using the same dataset for training and test, We estimated the transition quantity by using the superfamilies subset from the dataset SABmark, which is described in the section "Estimation of the Transition Quantity". We also used this dataset for optimizing the parameters Q  score [Q score is defined as the ratio of the number of correctly aligned residue pairs in the test alignment  to the total number of aligned residue pairs in the reference alignment. When all pairs are correctly aligned, the score have a maximum value 1, and when no-pairs are aligned the score have a minimum value 0. This score has previously been called the SPS (Sum of Pairs Score) [Ai  indicates the ith sequence of the reference alignment with length L, and let aik ∊ Ω* indicates the kth symbol in the sequence Ai. When aik ≠ *, it is important to find the number of the site in the test sequence corresponding to the symbol aik, whose numbers are denoted by nik. When aik = *, put nik = 0 . Then the Q score is given asAlignment accuracy was calculated with the y) score . The Q ss Score)  or the ds Score) . Let us ε, wopen, wextend, and the option controlling the output format. The program accepts only multiple-fasta format as an input format.The MTRAP algorithm is implemented as a C++ program. The program has been tested in several types of Linux machines including 32-bit ×86 platform and also has been tested on Mac OSX snow leopard (64-bit). The program has a number of command-line options, e.g., the option setting the value of a parameter such that We compared MTRAP with nine different alignment methods including the modified TCoffee by using all 1682 protein pairs of PREFAB 4.0 and all 630 protein pairs of HOMSTRAD. We used GONNET250 matrix with the MTRAP. The similarity between the test alignment  and the reference alignment (obtained from PREFAB 4.0 or HOMSTRAD) was measured with the Q score. Suppose as usual that the reference alignment is the optimal alignment, the results of PREFAB 4.0 , MTRAP has more than 80% accuracy , whereas Needle and ClustalW2 have less than 80% accuracy   calculation order for two input sequences with length m and n. Our CPU time shown in the Tables MTRAP has the similar calculation cost with other pairwise methods. That is, MTRAP has Pairwise sequence alignment is among the most important technique to perform biological sequence analysis, and is fundamental to other applications in bioinformatics. Any multiple sequence alignment that is gradually built up by aligning pairwise sequences is essentially based on high-quality pairwise sequence alignments. By modifying common multiple alignment method based on our algorithm as shown in this paper, the accuracy was improved significantly. Moreover, we think that our technique is applicable to not only global alignment, but also some others such as, local homology search and motif-finding, which will be our future works.Project name: MTRAPhttp://www.rs.noda.tus.ac.jp/%7Eohya-m/Project home page: Operating systems: Linux, UNIXProgramming languages: C++License: BSD licenseWe three  discussed all fundamental parts together. In details, mathematical idea mainly comes from MO and TH did mathematical algorithm. Moreover, TH and KS made computer algorithm and did computer alignment by means of this algorithm. All authors have read and approved the final manuscript.


Sequence-based phylogeny reconstruction is a fundamental task in Bioinformatics. Practically all methods for phylogeny reconstruction are based on multiple alignments. The quality and stability of the underlying alignments is therefore crucial for phylogenetic analysis.In this short report, we investigate alignments and alignment-based phylogenies constructed for a set of 22 ABC transporters using CLUSTAL W and DIALIGN. Comparing the 22 "one-out phylogenies" one can obtain for this sequence set, some intrinsic phylogenetic instability is observed — even if attention is restricted to branches with high bootstrapping frequencies, the so-called safe branches. We show that this instability is caused by the fact that both, CLUSTAL W as well as DIALIGN, apparently get "confused" by sequence repeats in some of the ABC-transporter. To deal with such problems, two new DIALIGN options are introduced that prove helpful in our context, the "exclude-fragment" (or "xfr") and the "self-comparison" (or "sc") option."One-out strategies", known to be a useful tool for testing the stability of all sorts of data-analysis procedures, can successfully be used also in testing alignment stability. In case instabilities are observed, the sequences under consideration should be carefully checked for putative causes. In case one suspects sequence repeats to be the cause, the new "sc" option can be used to detect such repeats, and the "xfr" option can help to resolve the resulting problems. Sinorhizobium meliloti alone, around 150 ABC-transporter proteins have been identified [in silico methods for structural and functional annotation. In view of some recent articles in which a close correspondence between the phylogeny and physiological function of these proteins was demonstrated , ,30, SinoHerein, we study the reliability of two standard multi-alignment programs, CLUSTALW and DIALIGN, by applying them to ABC transporters. We apply the "one-out strategy" to analyze the stability of the alignments constructed by these two methods, i.e., by removing – one by one – individual sequences from the input sequence set: Ideally, removing sequences from a sequence set should not affect the alignment of the remaining sequences; their alignment should agree with the subalignment derived for them by restricting the alignment constructed for the complete sequence set Fig. . But, asbootstrapping frequency, is then taken as a measure of the confidence one can have in this branch – or, more precisely, in the phylogenetic relevance of the corresponding split of the underlying sequence set induced by this branch. Hence, bootstrapping can be used to assess the reliability of a tree's branches derived from a fixed alignment and to identify its safe branches that we define as branches with a bootstrapping frequency above a certain threshold, but it does not reflect the reliability of the alignment itself. For this paper, we used a threshold value of 5/6.The most popular tool to test the reliability of a tree topology is the bootstrapping method. It works as follows ,6: for aIn this study, only the safe branches of a tree topology will be considered when comparing trees resulting from distinct alignments, and we will confirm that, as expected, instabilities concerning these branches indicate instabilities of the underlying alignments.exclude-fragment or, for short, the xfr option, allows the user to exclude certain pairs of sites, or pairs of segments, from being aligned to each other and, this way, to introduce expert knowledge into the alignment process as well as to explore interactively various alignment alternatives. In this regard, the xfr option is sort of "dual" to the recently introduced anchored-alignment option where the program is required to align certain user-specified parts of the sequences to each other – provided the requirements are consistent and there is at least some degree of similarity between the specified regions [guide tree used by CLUSTAL W for constructing CLUSTAL W alignments.Furthermore, we show that a new option that has been added in the context of our investigations to the DIALIGN software can lead to considerably improved alignments. This option, called the  regions ,27. In txfr option. This observation, while perhaps disturbing, but not of immediate relevance for the user, is apparently crucial for further development of this tool. Indeed, every multi-alignment approach has to address two major problems: (a) a procedure has to be defined that, in principle, associates a score to every possible alignment of any given sequence set and (b) an optimization algorithm has to be designed that finds optimal  alignments relative to that scoring procedure. Our study shows that DIALIGN can sometimes fail by some margin to find the maximum score indicating that improvements of its optimization procedures could lead to considerable improvement of the quality and, in particular, to "stabilization" of the alignments constructed by DIALIGN. Note that this result is not self-evident: According to the two parts of the alignment problem, there can be two reasons for mis-alignments as detected by instabilities: (i) the employed scoring function may itself be "unstable" , and (ii) the optimization algorithm produces alignments with scores significantly below the mathematical optimum. Our study shows that the latter problem actually plays a non-negligible role. Thus, it makes sense to develop alternative optimisation algorithms for the fragment-based alignment approach [As a by-product of our study, we found that the numerical values of the scores of multiple alignments produced by DIALIGN are sometimes noticeably below the scores of other multiple alignments that DIALIGN can be tricked to find for the same sequence set using the approach ,33,32.One reason for the instabilities of the alignments constructed for the set of ABC transporters under investigation is the repetitive structure of three of its sequences. In a last step, we will therefore decompose these sequences into non-repetitive parts by applying another new option of DIALIGN. For the resulting new sequence set, we will also compute alignments and phylogenetic trees. Remarkably, the DIALIGN algorithm produces perfectly stable alignments and phylogenies for this new data set.The paper is organized as follows: first, we shortly review the two multi-alignment programs CLUSTAL W  and DIALprogressive alignment method. That is, it reduces the computational costs of constructing a multiple alignment by reducing this task to a sequence of pairwise alignments of profiles representing previously calculated multiple alignments of subsets of the original sequence set. These pairwise alignments are calculated according to the classical Needleman-Wunsch algorithm [In our study, we investigated CLUSTAL W since it is still the most popular software propgram for multi-alignment. It is a classical implementation of the lgorithm  using a lgorithm ,13 or MUlgorithm ,7. Invesn sequences, and constructs a guide tree for the input sequences based on their pairwise similarity scores. To this end, it uses the neighbor-joining algorithm [slow-and-accurate mode in which also the score function itself depends on that guide tree. Generally, CLUSTAL W produces multiple alignments of high quality if the sequences are globally related and the degree of over-all similarity between them is sufficiently strong [A crucial question is, of course, to decide in which order the pairwise alignments are computed in a progressive approach. With default options, CLUSTALW calculates, in a first step, all lgorithm . The "pry strong ,16.fragment alignments or just fragments. Every possible fragment is given a score based on the probability of its random occurrence [consistent collection of fragments with maximal total score; consistency here means the combinatorial requirement that the selected fragments are required to fit into one single output alignment and do not 'cross' each other. Gaps are not penalized in this approach – and there is, in consequence, no worrying about how to quantify gap penalties.DIALIGN uses a completely different approach. The similarity between sequences is not measured by summing up, over all sites, properly specified individual similarity scores, but rather by comparing whole sequence segments of equal length with each other. Alignments are composed from gap-free segment pairs, so-called currence , and theFor pairwise alignment, the "best" chain of fragments is found using an exact recursive optimization algorithm ,22. For Note that CLUSTAL W and DIALIGN reflect very different philosophies regarding the definition of scores: DIALIGN uses a fixed scoring procedure whereas the scoring procedures used by CLUSTAL W depends -in the slow-and-accurate mode applied in our studies – on the guide tree used for the computation of the alignment. Thus, to compare two alignments of the same sequence set, one can compare their score if both are DIALIGN alignments whereas this would not make sense if both are CLUSTAL W alignments .Two recently implemented options of DIALIGN were used in this study:prevent certain pairs of sequence sites, or whole gap-free sequence segments, from being aligned with each other; we implemented this exclude-fragment (or xfr) option to improve the alignment of the ABC transporters in the present study. The xfr option can be used to prevent early inclusion of misleading fragments in the greedy alignment procedure outlined above, but also to explore interactively various alignment alternatives.• One way of influencing the output alignment is to self-comparison (or sc) option was implemented to search for repeats in a given sequence. To this end, a sequence is aligned with itself in such a way that any single sequence entry can be aligned to any other entry except itself.• The A and B that represent the ATP-decomposing domains whereas the physiological function of a third motif, the so-called signature sequence S, is still unknown. In [In this study, all investigations of ABC transporters – or, for short, ABCs – are based on the ATPase-domains of ABC transporters. Their primary structure is characterized by three highly conserved sequence motifs: two Walker motifs called nown. In , these sx stands for an arbitrary, and h for an arbitrary hydrophobic, amino acid. The order of these three motifs is – at least for the sequences considered in this study -where (from N- to C-terminus), with dashes representing arbitrary amino-acid sequences. Some of the ABCs comprise two sets of these motifs, so their structure isS1), sp| P32010| (S2), sp| P23199| (S3), sp| P08720| (S4), sp| P31220| (S5), sp| P25885| (S6), sp| P31134| (S7), sp| P26905| (S8), sp| P30750| (S9), sp| P15031| (S10), sp| P07109| (S11), sp| P09833| (S12), sp| P16678| (S13), sp| P30963| (S14), sp| P23888| (S15), sp| P29018| (S16), sp| P27299| (S17), sp| P33116| (S18), sp| P33951| (S19), sm| b20111| (S20), sm| b21260| (S21), sm| b20141| (S22). Here, tr, sp, and sm denote entries in TrEMBL, Swissprot , and in , respectively.More specifically, we will consider here a collection of 22 amino-acid sequences, i.e., the complete sequences of the following proteins: tr| Q9RK11|  of X, i.e., a bipartition of X into two disjoint non-empty subsets of X whose union is all of X while S(b) ≠ S(b') holds for any two distinct branches b, b' of T : {S(bi) : 1 ≤ i ≤ N} of N distinct splits of X that, in turn, are well known to determine T . Accordingly, if bootstrapping frequencies f(b) have been computed for every branch b of T, the set of all safe branches  at least 5/6) can be identified with the set of splits Σ safe(T): {S(bi) : 1 ≤ i ≤ N, f (bi) ≥ 5/6}.Let cf. e.g. ). Thus, T1 and T2 of X are said to be essentially concordant if each one "contains" the safe branches of the other, i.e., ifTwo phylogenies T1 and T2 are called definitively distinct.holds. Otherwise, Let us now assume that we are givenX of  sequences and• a collection Alg for computing phylogenies from sequences.• an algorithm X, let Alg.The sequence family ⅅ as aligned using CLUSTAL Wand DIALIGN. DIALIGN was used with default parameters and CLUSTAL W was run with the slow-and-accurate option. Subsequently, phylogenies were computed from the alignments using the neighbor-joining algorithm with the bootstrapping option of CLUSTAL W . This resulted in two definitively distinct phylogenies depicted in Fig. i consisting of all sequences in ⅅ except the sequence Si  in the same fashion. Each of these phylogenies was essentially concordant with (the restriction to the respective sequence sets of) one of the trees shown in Fig. 8, ⅅ10, and ⅅ22 led to Topology III trees  were between 140 and 559 "DIALIGN units" higher than the scores of the corresponding class-2 or class-4 alignments. In view of the fact that the DIALIGN scores are computed in terms of the logarithm of probabilities [In this subsection, we discuss which of the 5 alignment classes have the best DIALIGN score. To compute DIALIGN scores for all alignment classes, the bilities , these dxfr option of DIALIGN was used to exclude all fragments of class-2 and class-4 alignments that involve the subwords S2 or B2 of the sequences S20 or S21. As expected, this led to an alignment of class 5  and Topology III (DIALIGN) trees shown in Fig. Note that the restrictions of all CLUSTAL W and DIALIGN alignments to the sequences see Fig. . Thus, iThis demonstrates that the two different approaches towards improving the DIALIGN alignment, namely the exclusion of fragments and the decomposition of the repetitive sequences in N- and C-termini, do not lead to contradictory trees.S16 to S19. And they show that not all the problems concerning the CLUSTALW alignments of the ABCs are caused by repeats. Additionally, we analyzed the alignments and the trees of the sequence set ⅅ'': = {S1, ..., S19}: Again DIALIGN produced a stable alignment and a stable phylogeny that coincided with the restriction of a class-3 alignment and a Topology III tree to this subset, respectively, while the alignment as well as the phylogeny produced by CLUSTAL W still turned out to be unstable: The tree produced by CLUSTAL W for ⅅ'' coincides with the restriction of the tree derived for ⅅ'. But the sequence set consisting of S1–S6 and S8–S19, for example, leads to an alignment of class 3 and a Topology III tree.Furthermore, these analyses of the CLUSTAL W alignments and the trees derived from them explain in particular the stability of the branch containing the four taxa So far, we emphasized differences between tree topologies and the problems concerning their stability. In this section, we discuss which branches all our trees have in common and whether they are related to the physiological functions of the ABCs.S1, S10}, {S2, S4}, {S5, S6}, {S9, S11}, {S8, S13, S20, S21, S22}, {S8, S20, S21, S22}, {S20, S22}.All six tree topologies separate the following sequence clusters from the remaining sequences (often by safe branches): {S20, S21, S22 by the corresponding N- and C-terminal sequences S20N, S21N, S22N and S20C, S21C, S22C, the same holds for the two trees constructed for ⅅ' which, in addition, separate {S8, S20N, S21N, S22N} suggesting a a slower evolution of the N-terminal motifs of the sequences S20, S21, S22 compared with the evolution of the C-terminal motifs of these sequences.Furthermore, replacing each of the sequences S1 and S10 are iron transporters of Streptomyces coelicolor (S1) and E. coli (S10), S2 and S4 are related to the daunorubizin resistance of Streptomyces peucetius (S2) and Rhizobium leguminosum (S4), S9 and S11 transport amino acids in E. coli and S5 and S6 are ABCs from the MURA-RPON intergenic region of E. coli and from the NTRA/RPON 5' region of Rhizobium meliloti.The splits mentioned above are related to the physiological function of the ABCs: S13 is a phosphonates transporter of E. coli, whereas S8, S20, S21, and S22 are transporter of peptides: S8 is a dipeptide transporter of Bacillus subtilis, whereas S20, S21, and S22 are oligopeptide transporters of Sinorhizobium meliloti. Furthermore, the trees constructed for ⅅ' suggest that the common precursor of these five sequences differentiated into the phosphonates transporter S13 and the common precursor of the four peptide transporters S8, S20, S21, and S22. They also suggest that this precursor differentiated further into the dipeptide transporter S8 and the common precursor of the three oligopeptide transporters S20, S21, and S22 that evolved from this precursor by motif duplication which, in turn, differentiated further into S21 and the common precursor of S20 and S22 showing a stronger conservation of their N-terminus copy of the – A – S – B – motif than of their C-terminus copy. These parts of the sequences apparently were allowed to adapt to the new function more freely, suggesting that, after all, the class-5 alignments might, in spite of their lower DIALIGN score, be more appropriate than the class-3 alignments.Finally, a) to construct a multiple alignment of the input sequences in such a way that biologically related parts of these sequences are assigned to each other and (b) to construct a phylogenetic tree based on such an alignment. If these subtasks are to be solved algorithmically, two types of questions are to be addressed: suitable scoring functions need to be defined assigning quantitative confidence scores to each possible multi-alignment or phylogenetic tree, respectively. These scoring functions should be defined in such a way that biologically reasonable alignments or trees are more likely to receiving higher scores than biologically meaningless ones. Given such a scoring scheme, the second problem is an optimization problem, namely to find alignments or trees, respectively, with optimal  scores. Thus, phylogenetic tree reconstruction can be affected by two different types of errors. One kind of error reflects modeling problems such as unsuitable definitions of alignment scores or (dis)similarities for tree reconstruction. The other kind of errors are due to problems with finding the optimal alignment for a fixed alignment score and the optimal tree for a fixed (dis)similarity matrix. Both errors can, of course, produce instable alignments and phylogenies.Phylogenetic tree reconstruction from sequence data can be a very difficult task that involves two major subproblems, namely , analogous analyses are also required in the phylogenetic investigation of larger sets of ABCs – in particular, since quite a few ABCs have repetitive structure. Note that complete ABCs are composed of three kinds of domains, the ATPases, the transmembrane domains, and the periplasmatic binding domain. In the present study, we only considered repeats within the ATPases (see Section 3.1), but repeats can also occur in other domains.In this study, we obtained stable tree topologies after constructing new repeat-free sequences from our sequences using the sc option of DIALIGN. Using these new sequences, DIALIGN produced stable alignments whereas CLUSTAL W still produced unstable alignments, yet essentially stable phylogenies. An explanation for this observation  is perhaps that the threshold value 5/6 in the definition of safe branches is too high to reflect instabilities of the trees. Hence, in this case, one should actually use a lower threshold value, redo all analyses, and check whether such a refinement allows to find instabilities of the trees and to trace them back to the observed alignment instabilities. In our study, only a small set of ABCs was analysed. But, as will be shown in a forthcoming article optimal alignment score only implies that the corresponding alignment might be a noteworthy approximation to biological reality. In fact, it is not possible to define scoring functions for sequence alignment in such a way that score-optimal alignments necessarily coincide with biologically correct ones. For real-world sequences, common ancestry and common function and structure do not always correspond to similarity at the primary-sequence level. This is a fundamental limitation for all automated alignment approaches. To overcome these limitations, semi-automatic alignment methods have been proposed where expert knowledge can be used to guide the alignment procedure. The exclude fragments (xfr) option introduced in this article offers one way to support such semi-automatic approaches. An even stronger option is the previously introduced anchored alignment option where the program is forced to align certain user-defined parts of the sequences to each other, see for example [Note that the requirement of stability is only a  example .4.5. These safe branches were shown to separate ABCs with identical function from the rest where the function of an ABC was defined by the transported substrate. Thus, it can be assumed that such branches of phylogenetic trees can indeed be used for functional predictions as discussed in the introduction.Stability analysis in phylogenetic studies is helpful to learn in more detail which alignments and (dis)similarities may be better descriptions of various aspects of sequence evolution. E.g., if phylogenetic analyses with different tools consistently lead to the same stable results, then there is at least a good chance that these results reflect the correct branching pattern of the evolution of the sequences under consideration. For similar reasons, we recently introduced a tool that compares distinct multi-alignments of the same sequence data set . Such coNote that, in our analyses, ABCs from different organisms with identical functions were grouped together, but not ABCs from the same organism with different functions. As noted already in , this suin preparation).From our analyses, we may also postulate that the phosphonate transporters and the dipeptide and oligopeptides transporters have a common ancestor; yet, in contrast to the dipeptide transporters, the oligopepetide transporters have a repetitive structure. However, the data set here is too small to unravel their history in detail. In a forthcoming article, these aspects of sequence evolution will be considered in more detail (Wagner and Diaz,