Uncertainty in multiple sequence alignments has a large impact on phylogenetic analyses. Little has been done to evaluate the quality of individual positions in protein sequence alignments, which directly impact the accuracy of phylogenetic trees. Here we describe ZORRO, a probabilistic masking program that accounts for alignment uncertainty by assigning confidence scores to each alignment position. Using the BALIBASE database and in simulation studies, we demonstrate that masking by ZORRO significantly reduces the alignment uncertainty and improves the tree accuracy. Multiple sequence alignment is critical for many biological studies making use of sequence data. For evolutionary analysis, columns in multiple sequence alignments are hypothesized to contain homologous residues in different sequences. This is known generally as “positional homology”. The assignment of positional homology can be problematic, however. It follows that the quality of a sequence alignment has a large impact on the final phylogenetic trees A plethora of programs developed recently have led to significant improvement of the overall alignment accuracy Several metrics have been introduced to help assess the overall alignment quality A critical, yet largely unsolved problem in the field is how to assess the quality of the alignment at each individual position, . Knowing the quality of the individual position is important as “poorly” aligned columns are more likely to contribute noise than signal. Detecting and removing ambiguously aligned regions, a step known as masking and trimming in molecular phylogenetics, increases the signal-to-noise ratio and improves the discriminatory power of phylogenetic methods GBLOCKS is currently the most frequently used masking program that attempts to assess the quality of alignment position by position. It calculates the degree of conservation for each aligned position and then uses it to select conserved “blocks” for further analyses Several alternative masking programs were recently developed. For example, ALISCORE identifies low quality regions within protein alignments based on Monte Carlo resampling within a sliding window Here we introduce ZORRO, a probabilistic masking program that objectively measures the quality of the alignment at each individual position. ZORRO uses a pair Hidden Markov Model (pair-HMM) to model the sequence evolution and uses the model to calculate the posterior probabilities that residues of a column are correctly aligned or homologous. In this paper, we first introduce the theoretical motivation behind ZORRO. Then using simulated sequences, we demonstrate that ZORRO outperforms other masking programs in terms of the masking sensitivity and specificity. We further show that masking by ZORRO reduces the alignment uncertainty that in turn leads to more accurate phylogenetic trees for relatively long and divergent sequences.Our goal was to develop an objective metric that measures the confidence of the positional homology, the core assumption that underlies most molecular phylogenetic inferences. A sequence alignment is not an observation but rather a hypothesis wherein a particular alignment is selected as the best from multiple options. Consequently, when evaluating the quality of two aligned residues, it would be best to do it in the context of all possible alignments and not just one single alignment usually obtained. The idea is that if two residues are truly homologous, we assume that they will also align in most of the alternative alignments. On the other hand, if the match of the two residues is not reliable, then the likelihood that they appear together in the alignment space should be low. This type of strategy has been used empirically to extract good quality sub-alignments from a limited set of alignment variants and combine them into a new, often improved alignment Pair hidden Markov model (HMM) provides an ideal probabilistic framework for modeling alignment problems http://probmask.sourceforge.net.The details of the ZORRO algorithm and its implementation are described in the To evaluate the performance of ZORRO and other masking programs, we simulate protein sequence evolution using the program ROSE Our simulations were based on a representative set of 31 phylogenetic marker genes that are broadly conserved among bacterial species We used the simulated sequences to benchmark the performance of masking. After removing gaps from the true alignment, we realigned the sequences using MAFFT Given that ZORRO performs well relative to the other masking programs, our next question was to determine if it produced output that could then be used to clean up alignments with reduced uncertainty. This would be beneficial because as discussed above, uncertainty in alignments can lead to phylogenetic inference errors. Ambiguous columns typically do not persist between different alignment treatments and will cause alignments to differ. By measuring the extent of differences between alignments, the uncertainty in alignments can be quantified. The head-or-tail approach discussed earlier is a simple and elegant way of assessing the alignment uncertainty We realigned the 216 protein families in the BALIBASE database from both directions (head or tail) using the program MAFFT. BALIBASE consists of protein families of various size, sequence length and levels of similarity and covers a wide range of structural folds and distinct patterns of evolution histories. For each protein family, we then calculated the proportion of residue pairs that were paired differently (discrepancy fraction) in the head vs. the tail alignments, and used it as a measurement of alignment uncertainty, with or without ZORRO masking. Simulation studies provided further support that ZORRO benefit the downstream phylogenetic reconstructions . For theUnder the conditions tested in our study, ZORRO consistently outperformed GBLOCKS, ALISCORE and GUIDANCE in NJ trees. GBLOCKS, ALISCORE and GUIDANCE improved the NJ trees when the alignments were relatively long  but had little or even negative impact when the alignments were short. For ML trees, ZORRO outperformed GBLOCKS and ALISCORE and was on a par with GUIDANCE. One advantage of using ZORRO and GUIDANCE is that they assign a confidence score to each individual position, so it is possible to weigh each alignment position by their confidence scores . Low-quality positions will still contribute to phylogenetic reconstruction but will do so at a much lower weight. In comparison, position-specific weighting is not possible with GBLOCKS and ALISCORE. Consequently, low-quality positions are removed prior to the phylogenetic reconstruction, which can lead to the excessive loss of phylogenetic signal.rplL, 126; nusA, 466; rpoB, 1291) and evolved them at two different substitution rates  in the alignment iC and jC in a column C, we can calculate the posterior probability Pr [Ci◊Cj] that the two residues are aligned under the model:iC is aligned to jC, whereas the denominator is the probability of all alignments between iS and jS. For each pair of sequences iS and jS in the alignment, these terms can be calculated efficiently for all pairs of residues (and gaps) in quadratic time by using the forward and backward algorithm To estimate the probability that two residues in a column are aligned correctly, ZORRO implements a variant of standard pair HMM with the state space as described in To score a column in the alignment, the pair-wise posterior probabilities are combined using a weighting scheme described below. The general sum of pairs scheme is not directly suitable for calculating the column score because sequences are not equally related  for a column C, which is the weighted sum of the pair-wise homology probabilities:The weight 2L2),O/2 pairs of sequences. The number of pairs in R is constrained to be at least N and can be specified by the user. The confidence score S(C) of each column C is calculated by using weighted sums of pairs formula:2)O as the representative, resulting in using a total of 10 marker genes  in our simulation study. The site-specific substitution rate for each gene was estimated using the program rate4site We used the program ROSE Simulated protein alignments were used to benchmark the performance of the masking programs. After removing the gaps from the simulated alignments, the sequences were realigned using the MAFFT program MAFFT sequence alignments, with or without masking, were used to reconstruct neighbor-joining (NJ) trees using PHYML Figure S1The 100-taxon tree used to guide the protein sequence simulations. This tree is derived from the ‘genome tree’ of 720 bacterial species (TIF)Click here for additional data file.File S1This file contains all the control files used by ROSE to simulate protein sequence evolution for the ten marker genes.(DOC)Click here for additional data file.


In this paper, we introduce a progressive corner cutting method called Reticular Alignment for multiple sequence alignment. Unlike previous corner-cutting methods, our approach does not define a compact part of the dynamic programming table. Instead, it defines a set of optimal and suboptimal alignments at each step during the progressive alignment. The set of alignments are represented with a network to store them and use them during the progressive alignment in an efficient way. The program contains a threshold parameter on which the size of the network depends. The larger the threshold parameter and thus the network, the deeper the search in the alignment space for better scored alignments.http://phylogeny-cafe.elte.hu/RetAlign/We implemented the program in the Java programming language, and tested it on the BAliBASE database. Reticular Alignment can outperform ClustalW even if a very simple scoring scheme  is implemented and merely the threshold value is increased. However, this set-up is not sufficient for outperforming other cutting-edge alignment methods. On the other hand, the reticular alignment search strategy together with sophisticated scoring schemes  overcome FSA and in some accuracy measurement, even MAFFT. The program is available from Reticular alignment is an efficient search strategy for finding accurate multiple alignments. The highest accuracy achieved when this searching strategy is combined with sophisticated scoring schemes. Therefore, the in silico prediction of protein structures is still demanding, and the majority of the protein structure prediction methods need accurate alignments. There are two major technical hurdles in the multiple sequence alignment problem. The first is the scoring problem: how to score the alignments such that the best scored alignment is the most accurate one. The second is the algorithmic problem: how to find the best scored alignment.The multiple sequence alignment problem is still the Holy Grail of bioinformatics . There aO(NL) memory and running time. It has been proven that finding the best scoring multiple alignment under the sum-of-pairs scoring scheme is NP-hard [Significantly more effort has been put into the research for solving the second challenge. Although the number of possible alignments of two sequences grows exponentially with the length of the sequences, finding the best scoring alignment of two sequences is computationally feasible, since such an alignment can be found by iteratively comparing the prefixes of the two sequences . The opt NP-hard , therefoa priori estimated upper limit for the score of the optimal alignment to define such a strip [et al. obtained a strip around the parsimony-based optimal alignment for HMM-based calculations [on the y using the so-called diagonal extension method [The memory requirement and running time can be reduced by corner-cutting methods. Corner-cutting algorithms define a narrow strip in the dynamic programming table which contains the optimal alignment. Some methods use an  a strip -7. Hein ulations . The strn method . The corn method ,11, withn method . HoweverAbove exact methods, approximation methods for the multiple sequence alignment problem are also widespread. The most commonly used approximation to multiple sequence alignment is the progressive alignment approach -17, whicThere have been successful attempts in other directions to reduce the computational time required to align sequences. MAFFT employs the Fast Fourier Transformation (FFT) technique to rapidly identify homologous regions by converting the amino acid sequence into a sequence of volume and polarity values . The twoOne artifact shared by all of the previously mentioned methods is that evolutionary events are scored using user-specified values . The accuracy of the alignments largely depends on the selection of these parameter values. To overcome these difficulties the statistical alignment approach has been introduced where evolutionary models describe the type of events that transform the sequences and provide a means of calculating the probability of a sequence of events. The alignment is then produced in an optimisation framework such as maximum likelihood or Markov chain Monte Carlo by finding the set of events explaining the evolution of the sequences with a high probability and the parameters of the evolutionary model are estimated from the data. This approach is taken by computationally expensive methods such as BAli-Phy  and Statk -2 ways to split an alignment column containing k characters into two columns with gaps. Any alignment containing such pair of alignment columns will be a neighbour of the high-scored alignment in the dynamic programming table. Furthermore, the score of these alignments will be Significantly lower than the score of the high-scored alignment, since the scores differ in two gap scores and the scores missing due to not aligning homologous amino-acids.In this paper, we introduce a novel corner-cutting method combined with progressive sequence alignment. Unlike former corner-cutting methods, our method does not define a compact part of the dynamic programming table to be filled in. The rationale behind the idea is the following. It is easy to see that any high-scored alignment is surrounded by a large set of low-scored alignments, and the number of low-scored alignments increases exponentially with the number of sequences. Indeed, a high-scored alignment contains several alignment columns containing homologous amino-acids. There are 2Instead of defining a compact part of the dynamic programming table, our approach stores a set of optimal and suboptimal alignments at each step of the progressive alignment procedure. At an internal node of the guide tree, the two sets of alignments of the two children nodes are aligned against each other. We use a special data structure for both representing the alignments and aligning the set of alignments against another set of alignments. The common parts of the alignments are represented only once, and aligned only once, thus saving a large amount of memory and running time. The alignments in our representation form a reticulated network .• alignments ending in two aligned (matched) characters. The score of the best alignment of prefixes jb. The score of the best alignment of prefixes i Aand j Bin this set is denoted by f I.• alignments ending in an insertion of character ia. The score of the best alignment of prefixes i Aand j Bin this set is denoted by f D.• alignments ending in a deletion of character i Aand j Bis then max{f M, f I, f D}. f M, f Iand f Dcan be efficiently calculated using dynamic programming. The DP tables are initialised as:The score of the optimal alignment of prefixes The dynamic programming recursion then goes from shorter prefixes towards larger prefixes in the following way:i/-j adenote the alignment column showing deletion of i ain an alignment in which the first character in sequence B to the left of this alignment column is jb. Similarly, let i/bj -denote the alignment column showing the insertion of character j bin an alignment in which the first character in sequence A to the left of this alignment column is ia. The backward algorithm calculates the following three types of entries:The backward-align algorithm is more sophisticated. Let bM denotes the score of the best alignment of suffixes i Aand j Bwhose score is calculated as if the i/bj aalignment column was before it. Namely, if the alignment starts with a gap, it will be scored with the gap opening penalty.1. bI denotes the score of the best alignment of suffixes i Aand j Bwhose score is calculated as if the -i/bj alignment column was before it. Namely, if the alignment starts with an insertion, it will be scored with the gap extension penalty.2. bD denotes the score of the best alignment of suffixes i Aand j Bwhose score is calculated as if the i/-j aalignment column was before it. Namely, if the alignment starts with a deletion, it will be scored with the gap extension penalty.3. These backward alignment scores can also be computed using a dynamic programming approach similar to the forward case. The initialisation of the backward DP tables is:The DP tables are then filled from the shorter suffixes towards the longer suffixes, that is, backward on the indices. The recursions:Using the forward and the backward scores it is possible to find all alignment columns that appear in an alignment with a score above a given threshold. This is based on the following theorem:i/bj  is Mf  + bM f  + bI, f D + bD.) ia/jb, then cutting the alignment after this alignment column will create two alignments. The left one is an alignment of prefixes i Aand j Bin which the last alignment column is ia/jb. The right one is an alignment of suffixes Ai and Bj whose score is calculated by adding the alignment column ia/j bbefore it. The best scored alignment containing ia/j bare cut into the best scored left and right alignment, by definition, with scores f M and bM. The score of the alignment is the sum of these two values.We give a proof for the first case, the proof for the other two cases goes in the same way. If an alignment contains b(α). We define the x-network of the alignments in the following way.Theorem 1 provides the means to collect the alignment columns that participate in an alignment having score above a given threshold. The best score of the alignment column will be denoted by A and B, x ≥ 0, the x-network of the alignments of A and B is a directed graph G. The vertex set consists of alignment columns α for which b(α) ≥ opt - x, where opt is the score of the optimal alignment of A and B; plus two auxiliary vertices, representing the beginning and the end of the alignment. These two auxiliary vertices are denoted by Start and End. An edge is going from vertex α1 to vertex α2 if there is an alignment in which α1 is followed by α2. The outgoing edges from the Start vertex go to the alignment columns with which the alignment might start, and the incoming edges of the End vertex come from the alignment columns that might be at the end of an alignment.For any sequences x-network never contains dead ends.The following theorem states that an For any sequences A, B, x ≥ 0, and α vertex of the x-network, there is a directed path from Start to α and also from α to End.α is in the x-network, b(α) ≥ opt - x. Consider an alignment containing α with score b(α). Any α' of this alignment has a best score greater or equal than b(α), hence they are all in the vertex set of the x-network. This alignment defines one possible directed path from Start to α and also from α to End.Since x-network can be constructed using an algorithm that first runs the forward and backward algorithm to calculate b(α) for each possible alignment column α, selects those columns for which b(α) ≥ opt - x, and builds the network from them.An We are going to extend the Waterman-Byers algorithm to align a network of alignments to another network of alignments. First we define the network of alignments.A1, A2, . . . kA, k ≥ 1 is a directed acyclic graph whose vertices are alignment columns of the set of sequences together with a unique source (denoted by Start) and a unique sink (denoted by End). The vertices along any path from the source to the sink form a multiple sequence alignment of the set of sequences.A network of alignments of sequences x-network is a network of alignments. Moreover, any single sequence (meaning k = 1) can be considered a simple, formal network. In that case, the formal alignment columns contain only one character, and the network is a single line containing only one alignment. We can generalise the definition of the x-network of two sequences to the x-network of two networks of alignments. For this, we first have to define the alignment of alignments.Obviously, an A1, A2, . . . k Aand B1, B2, . . . l Bis a multiple sequence alignment of these k + l sequences such that the non all-gap columns of the first k rows gives back l rows gives back ℬ.An alignment of two alignments k rows or in the last l rows, we indicate what was the previous non all-gap alignment column from ai/-j indicates an alignment column in which the first k row is the th ialignment column from l rows contain only gaps, and the first alignment column to the left of this alignment column in which the last l rows contain not only gaps is the th jalignment column from ℬ. In the next definition when we talk about alignment columns from the multiple alignment of all As and Bs sequences, we always mean alignment columns containing this additional information. Similarly, from now on, we always assume that the alignment is an alignment of two alignments, the first containing k lines, the second containing l lines.When we take an alignment column containing all-gap characters in the first A and B and x ≥ 0, the x-network of A and B is a directed graph G. The vertex set consists of two auxiliary vertices, representing the beginning and the end of the alignment and all alignment columns α for which b(α) ≥ opt - x, where b(α) is the maximal score of the alignment that can be achieved by aligning an alignment A to an alignment ℬ ∈ B so that it contains the column α. opt is the maximal score that can be achieved by aligning any alignment A to any alignment ℬ ∈ B. An edge is going from α1 to α2 if there is an alignment in which α1 and α2 are neighbour columns. The outgoing edges from Start go to the vertices that might be the first alignment column in an alignment, and the incoming edges of the End vertex come from the vertices that might be the last alignment column in an alignment.For any two networks of alignments When we align a network to a network using a dynamic programming algorithm, it is important to visit the alignment columns of the network in an order such that the entries are already calculated by the time we want to use them in the dynamic programming recursion. Therefore we introduce the linear extension of networks that can be used for traversing the network.v and u, if there is a directed path from v to u then v < u.A linear extension of a directed acyclic graph is a total ordering, <, on the vertices such that for any two vertices Furthermore, the forward-align and the backward-align algorithms work with prefix-alignments and suffix-alignments defined in the following way.A to an alignment ℬ ∈ B. Similarly, a suffix-alignment is a suffix of an alignment achievable by aligning an alignment A to an alignment ℬ ∈ B.A prefix-alignment is a prefix of an alignment achievable by aligning an alignment x ≥ 0 and a couple of networks of alignments, A and B, together with a linear extension for each network. The output is the x-network of A and B together with a linear extension of it.The generalisation of the Waterman-Byers algorithm is the following. The input consists of a threshold value k lines is i aand in the last l lines is j bfor each subset of alignments:The algorithm uses a forward and a backward dynamic programming algorithm. The forward align algorithm calculates the score of the best prefix-alignment in which the last non all-gap columns in the first ia/jb. The score will be denoted by f M.• alignments ending with i-/jb. The score will be denoted by f I.• alignments ending with ia/j-. The score will be denoted by f D.• alignments ending with The initialisation is:The dynamic programming algorithm visits the vertices of the two networks in their linear order. The recursions are:i) is the set of indices of vertices sending an edge to the vertex indexed by i, and g is the gap penalty function for alignment column b/d preceded by alignment column a/c. We assume that the gap penalty for a given alignment column can be calculated from the alignment column in question and its preceding alignment column. See details in the subsection Gap penalties below. The maximum of an empty set is defined to be - ∞.where The backward algorithm calculates the following scores:bM denotes the score of the best suffix-alignment that can follow the alignment column ia/jb. Furthermore, the gap score of the first alignment column is calculated as if i/bj awas inserted before the first alignment column.• bI denotes the score of the best suffix-alignment that can follow the alignment column i-/jb. Furthermore, the gap score of the first alignment column is calculated as if i-/j bwas inserted before the first alignment column.• bD denotes the score of the best suffix-alignment that can follow the alignment column ia/j-. Furthermore, the gap score of the first alignment column is calculated as if ia/j -was inserted before the first alignment column.• The initialisation of the dynamic programming algorithm isEndA) and EndB) are the sink vertex of networks A and B, respectively.where The dynamic programming algorithm visits the vertices of the two networks backward in their linear extension. The recursions arei) is the set of indices of vertices to which an edge is going from the vertex with index i. Similarly to Theorem 1., it is true that the best score of alignments containing ia/jb, i-/j band ai/j -is f M + bM, f I + bI and f D + bD, respectively. Therefore, the pair of indices  is visited in lexicographical order, and those alignment columns α = ia/j bor i-/j bor ai/j -are selected for which b(α) ≥ opt -x. The maximal score, opt, can be calculated from the following equationwhere x-network.Similarly to Theorem 2., it is easy to show that there are no dead ends in the so constrained network. The following theorem states that visiting the alignment columns in lexicographical order will provide a linear extension for the constructed iThe lexicographical ordering of alignment columns together with arbitrary ordering of a/j, -ib/j and aib/-j is a linear extension for the x-network of networks A and B if the indices i's and j's are linear extensions for the networks A and B, respectively.ia/j bmight be i'/bj'a, i'/bj' -or i'/-j' afor some i' ∈ i) and j' ∈ i). Since indices in A are linear extensions, i' < i for any i' ∈ i), and thus, in the lexicographical order, all possible preceding alignment columns are smaller than ia/jb.The preceding alignment columns for i-/j bmight be i/bj'a, i/bj' -or i/j' afor some j' ∈ j). Since indices in B are linear extensions, j' < j for any j' ∈ j), and thus, in the lexicographical order, all possible preceding alignment columns are smaller than i-/jbThe preceding alignment columns for i/-j amight be i'/bja, i'/bj -or i'/-j afor some i' ∈ i). Since indices in A are linear extensions, i' < i for any i' ∈ i), and thus, in the lexicographical order, all possible preceding alignment columns are smaller than i/-ja.The preceding alignment columns for The Reticular Alignment algorithm is the following:1. Build or load a guide tree for the sequences2. Transform the sequences at the leaves of the guide tree into simple 'linear' networksv with children u1 and u2, labelled with the networks of alignments 1 Aand 2A, respectively, calculate the vx-network of 1 Aand 2 Ausing the generalised Waterman-Byers algorithm3. Visit the internal nodes of the guide tree in reverse traversal order. For each internal node x-network calculated at the root of the guide tree.4. Return the best scored alignment from the x is set to 0, only the  optimal multiple alignments are stored in the x-network. In this case, the Reticular Alignment algorithm mimics a standard progressive alignment method. When x is set to ∞, the Reticular Alignment method performs an exhaustive search in the space of multiple alignments, namely, it finds the best scored alignment. As x increases, the size of the network also increases, having a similar effect on the running time and memory usage. Along with the x value the Reticular Alignment algorithm can be parameterised in a list of ways:When • guide tree construction method• similarity scoring of alignment columns• gap scoring model and gap penalties• strategy to select threshold values at the internal nodesHere we briefly describe the choices we had and the decisions we made considering these aspects of the algorithm.Standard methods for constructing a guide tree using pairwise comparisons of the input sequences include UPGMA and Neighbour Joining (NJ) ,26. We iThe NJ algorithm generates an unrooted tree. Because RetAlign requires a rooted tree that can be traversed from the leaves upwards, we root the tree using the 'mid-point' method as described in . The comL2) to Θ(L) where L is length of the longest sequence.In a future version of RetAlign we plan to implement a basic optimisation such as the Hirschberg algorithm  to reducIt is well known that affine gap penalties  generate Significantly more accurate pairwise alignments than linear gap penalties. Therefore it seems reasonable to define similar gap penalties for multiple sequence alignments. One natural way is the sum-of-pairs scoring with affine gap penalty, when one generates all pairwise alignments from the multiple alignment, removes all-gap columns, scores the so-obtained alignment using affine gap penalty, and sums these scores over all pairwise alignments. Surprisingly, finding the best sum-of-pairs scored multiple alignment between two multiple alignments is NP-complete . The heuconsistent in that it does not depend on the order of the sequences within the selected pairs (the indel matrix is symmetric)• symmetric - the reverse of a multiple alignment has the same score • • best approximate the sum-of-pairs scoresThe simplest case to consider is when there is an insertion in one of the sequences: OP + 2EX where OP and EX are the gap opening and extension penalty. This - and any similar cases where the length of the insertion is different - can be mimicked precisely if (and only if) the score of the pattern EX while the score of OP/2 due to the symmetric property. The score of OP for similar reasons as it starts a new sequence of gaps in both directions. With this choice cases such as EX/2 suffices for (1) and OP/2 does for   so the total scores of EX, OP and OP/2 are obtained that match the scores of the corresponding patterns formed by removing the gap-only columns). We resolved the ambiguity by choosing OP/2 as the score because we expected this to provide the best approximation of the sum-of-pairs scores with a systematic overscoring in cases such as (1). Other alternatives include EX or EX/2, both of which have been later shown to yield slightly lower overall accuracy as measured on the BAliBASE reference database.The sum-of-pairs indel score of this alignment is In addition to RetAlign's default pairwise indel score model presented above we also implemented the simplified, non-pairwise indel scoring method used in ClustalW. In this scheme, when two sets of sequences are aligned, each insertion or deletion of a full alignment column receives a single gap penalty - even if the alignment column contains several gaps. This score can be computed considerably faster  but creates anomalies when suboptimal alignments are inserted or deleted: the gaps 'hidden' in the columns of the suboptimal alignments are not penalised and these columns become overly represented in the final alignment (see results). Unlike the pairwise indel scoring this score cannot be used as an accuracy measure of multiple alignments because it depends on which sets of sequences are being aligned in the last step.We score substitutions in accordance with the sum-of-pairs scoring scheme. A similarity score is computed for each alignment column as the sum of similarity values for each pair of non-gap characters in the column . The siThis pairwise scoring method is slightly different from ClustalW's approach where the substitution score is dependent on what the two sets of sequences are that are being aligned: only 'cross-scores' are taken into account (scores for pairs of non-gap characters where the first element of the pair is from a sequence in the first set and the second from the second set). The similarity score of columns with insertion or deletion in the ongoing alignment is thus zero. We also implemented this modified similarity scoring method and combined it with the non-pairwise indel scoring shown in the previous section to imitate ClustalW's scoring model.We introduced the indel and similarity scoring models of RetAlign in the last two sections. The total  score of a multiple alignment as produced by RetAlign is the sum of the indel and similarity scores, both of which are calculated pairwise. Though very similar, this score slightly deviates from the sum-of-pairs score as a result of the approximation of the indel score using adjoining alignment columns only - this is explained in detail above. In practice we did not encounter any situation where the difference was Significant and the optimisation targeted to maximise the internal score efficiently boosted the sum-of-pairs score, too. Both scores are completely independent from the phylogenetic tree connecting the sequences and can be used as an accuracy measure.One inherent weakness of the sum-of-pairs scoring, though, is that evolutionary events separating a distant sequence from a number of closely related (overrepresented) sequences are overscored - one evolutionary event in time might be penalised several times in the alignment score. To overcome this, we introduced sequence weighting, based on principles set out in ClustalW. First, a list of weights is calculated and assigned to sequences using the topology and edge lengths of the guide tree, precisely as described in . The weix value for each alignment step at the internal nodes. We chose to set the x value dynamically such that the final size of the alignment network is at most (t + 100)% of the length (number of columns) of one of the best scored multiple alignments, where t is a threshold parameter set by the user. Note that this is the "Threshold to be passed for computation" value on the GUI of our application. The slider underneath is just for convenience, with a log transformation. For any such t value, the corresponding x value can be found by building the network gradually: at first, alignment columns are placed in a priority queue where the key is the score of the best alignment they appear in; then groups of columns having equal score are removed from the queue iteratively, starting with the ones having the highest score, and added to the growing network en masse while the size limit permits.The size of the alignment space that is being explored by the Reticular Alignment algorithm and consequently, the accuracy of the alignments created depends on the strategy for choosing an x was constant or set to a fixed proportion of the optimum score because the later would have an unpredictable effect on the running time and memory usage and could also cause the alignment networks to vary considerably in relative size at the internal nodes. In contrast, with our method, the memory requirement can be estimated from t and the proportion of the number of suboptimal alignment columns to optimal columns is uniform over the whole tree.This approach is more advantageous than if the The scoring scheme of RetAlign involves summing similarity values and gap penalties for all sequence pairs. These pairwise summations are carried out repeatedly on alignment columns (for similarity) and pairs of columns (for gap scores) to fill each element of the dynamic programming tables. The straightforward implementation can thus have a huge impact on the running time when many sequences are aligned. For this reason we developed techniques to speed up these calculations.v1, . . . nv, pairwise pattern scores must be summed for all pairs: The two problems are essentially the same: given a list of pattern values w1, . . . n ware given in addition to the patterns then n + 16 steps. In the similarity score case, however, we chose to store a list of  pairs so that the calculation takes only n + l2 steps where l is number of different character types present in the column. Though this requires one or two additional table lookups per iteration, the savings are huge when there are conserved columns made up of just a few different characters.We implemented the Reticular Alignment method in the Java programming language with all features for the choices of parameters as described in the previous section. The method was tested on the BAliBASE database , and comBAliBASE provides a scoring tool  to measure the accuracy of sequence alignments based on the reference alignments in the database. This tool offers two accuracy measures (SP and TC) and allows assessment based on either all or a subset of alignment columns thus essentially giving four different accuracy scores. SP is the number of correctly aligned residue pairs divided by the number of aligned residue pairs in the reference alignment, TC is the number of correctly aligned columns divided by the number of columns in the reference alignment. SP and TC can also be calculated on columns of the core blocks only - these feature columns are described in the BAliBASE database by separate files. We denote the so-obtained scores 'Feature SP' and 'Feature TC'. All four of these scores can be regarded as a sensitivity measure (in classiffcation terminology) because characters/columns incorrectly shown homologous do not decrease the score.We measured the accuracy of the Reticular Alignment method on BAliBASE v1.0 and v2.0 Ref1-5 datasets and compared it to the performance of the well-known alignment software ClustalW, MAFFT and FSA. See results in Figure t parameter, namely, how much the accuracy can be improved by a deeper search in the alignment space. Some of our findings are quite surprising, discussed in the following subsections.We were interested in how the accuracy of our method depends on different parameters. Since the parameter space is four dimensional  with several choices along each dimension, we do not show the results for each possible combinations of parameters. Two parameters  influence only the score of the multiple alignments, one parameter  influences how to search in the search space, and one parameter (how to build the guide tree) influences the search strategy , and might also influence the way of scoring alignments (if the sequences are weighted by the guide tree). For each fixed score function, we tested how the alignment accuracy changes with the t parameter on the alignment accuracy. The average alignment accuracy improves as the t parameter increases, see Figure t1, the best alignment at internal node a vhas a score as, and an ax,1-network is generated at av. From this network, the best alignment at some ancestral node of av, denoted by b vhas a score sb,1, and an bx,1-network is generated at bv. From this network, the best alignment at the root of the guide tree has score sr,1. Now we change the threshold to t2. The best alignment at node a vis still the same, but we create a larger, xa,2-network at this node, xa,2 >xa,1. From this network, we are able to find a better scored alignment at node bv, which has score sb,2 >sb,1. We build up an xb,2-network around this node, but it might happen that sb,2 - xb,2 >sb,1. This means that the new xb,2-network does not contain any alignment from the old, xb,1-network! Therefore, the progressive alignment method with reticular threshold t2 will operate on a set of alignments above the node b vwhich are completly different from the set of alignments appeared during the progessive alignment method with reticular threshold t1. The consequence is that the best alignment at the root obtained from the xb,2-network might have a score sr,2 <sr,1. Namely, the score of the final alignment might decrease with increasing the reticular threshold parameter.As the main novelty of our method is the sophisticated search for the best scored alignment, we first show the effect of the t threshold value is shown on Figure t. However, these examples were relatively rare. Hence, the slight occasional decrease in the accuracy caused by the increase of t is mainly due to the non-perfect correlation between the RetAlign's internal score of an alignment and the alignment accuracy measured on BAliBASE. The most interesting case is discussed in the next subsection.To test the second hypothesis, the internal score of the alignments were measured, i.e. the score that the Reticular Alignment algorithm was to maximise. The dependency of this internal score on the Clustal uses a simple non-pairwise gap-penalty for multiple alignments as described in the Methods section. This seems a rational choice for Clustal, as this gap scoring scheme indeed generates better alignments for Clustal than the pairwise scoring scheme.t parameter when the alignments are scored using a pairwise gap penalty scheme yielded an increase in the accuracy of the generated alignments, and eventually the Reticular Alignment method with this gap-penalising scheme overtakes ClustalW, see Figure However, when we extend the scope of the search in the alignment space, and keep not only the locally optimal alignment during the progressive alignment procedure, we see a different picture. Increasing the t parameter, see Figure On the other hand, when simple non-pairwise gap penalties were applied, the accuracy decreased with increasing the Although Reticular Alignment outperformed ClustalX with a simple sum-of-pairs scoring scheme, and without any sophisticated gap scoring scheme, its performance with the less sophisticated scoring schemes was worse than the performance of the cutting-edge multiple sequence alignment methods. Therefore, we improved the scoring scheme both for similarity scoring and for gap scoring.It is well-known that the relative difference between the score of the fully conserved alignment column and the score of the alignment column with a single mismatch decreases with the number of sequences . This arSince our sequence weighting method uses the guide tree, it is also important to construct a good guide tree. We found that NJ outperforms UPGMA measured in alignment accuracy (data not shown). Since the NJ algorithm generates an unrooted tree, and the Reticular Alignment method needs a rooted tree, the NJ tree must be rooted. Changing the root of the guide tree also changes the progression of the multiple alignment. The more balanced the tree, the closer the numbers of sequences in the two alignment networks. We found that balanced trees generated by the 'mid-point' method as described in  generateFinally, it is also important to distinguish gap scores based on whether hydrophilic or hydrophobic amino-acids are inserted and/or deleted. Applying the same scoring scheme that ClustalX uses improved the alignment accuracy.Fortifying the Reticular Alignment method with these sophisticated scoring schemes yielded a method that generated highly accurate alignments. Reticular Alignment outperformed all of ClustalX, MAFFT and FSA in SP values on the BAliBASE v1.0 database, and only MAFFT outperformed Reticular Alignment in the TC values, see Figure As the threshold value increases, the size of the alignment network will increase, too. Figure Previous corner-cutting methods define a compact part of the dynamic programming table for searching the best scored alignment. These methods become very inefficient when the number of sequences increases. We introduced a new progressive alignment method called Reticular Alignment, which obtains a set of optimal and suboptimal alignments at each step of the progressive alignment procedure. This set of alignments is represented by a network and are not directly embedded into the high-dimensional dynamic programming table. The set typically contains high-scored alignments that are usually not neighbours in the dynamic programming table (see, for example, the already mentioned Figure This novel corner-cutting approach allows the efficient search of the space of multiple sequence alignments for high-scored alignments. The method has a parameter which affects how much of the alignment space is explored. Furthermore, the Reticular Alignment method can be combined with any scoring scheme, and in this way, we were able to infer what is the importance of sophisticated scoring schemes and more exhaustive searches in finding accurate multiple sequence alignments.The conclusion is that it is important to increase the search space for finding high-scored alignments. The Reticular Alignment method could find more accurate alignments than ClustalW even when the gap-scoring scheme was Significantly less sophisticated than the scoring scheme of ClustalW. For example, ClustalW gives different gap scores for hydrophilic and hydrophobic amino acids. This is considered to improve the alignment quality as hydrophobic amino acids are on the surface of globular proteins forming loops, and these loops undergo Significantly more insertion and deletion events than other parts of the proteins. Still, Reticular Alignment could generate more accurate alignments than ClustalW by merely extending the search space and without applying the above mentioned sophisticated scoring scheme of ClustalW. On the other hand, sophisticated scoring schemes are also necessary to get highly accurate multiple alignments. Combining sophisticated scoring schemes with the Reticular Alignment progressive alignment approach yielded a method whose accuracy is comparable to that of cutting-edge alignment methods. Without such sophisticated methods, the Reticular Alignment method only outperformed the ClustalX method, and were beaten by MAFFT and FSA in all accuracy measurements. Therefore it is also an important question how to find the scoring function that provides the most accurate multiple alignments. Kececioglu and Kim gave a fast linear programming-based method that finds parameter values that make given example alignments be optimal-scoring alignments of their strings . Such exIM proposed the extension of Waterman-Byers algorithm for aligning a network of alignments to a network of alignments, and implemented a prototype. AS developed the majority of the current RetAlign implementation. ÁN proposed the data structures and algorithms for efficient score calculation, and created the benchmarking framework to compare alignment programs. JH encouraged the discussions. All authors read and approved the final manuscript..


Transcriptome sequencing (RNA-Seq) overcomes limitations of previously used RNA quantification methods and provides one experimental framework for both high-throughput characterization and quantification of transcripts at the nucleotide level. The first step and a major challenge in the analysis of such experiments is the mapping of sequencing reads to a transcriptomic origin including the identification of splicing events. In recent years, a large number of such mapping algorithms have been developed, all of which have in common that they require algorithms for aligning a vast number of reads to genomic or transcriptomic sequences. Although the FM-index based aligner Bowtie has become a de facto standard within mapping pipelines, a much larger number of possible alignment algorithms have been developed also including other variants of FM-index based aligners. Accordingly, developers and users of RNA-seq mapping pipelines have the choice among a large number of available alignment algorithms. To provide guidance in the choice of alignment algorithms for these purposes, we evaluated the performance of 14 widely used alignment programs from three different algorithmic classes: algorithms using either hashing of the reference transcriptome, hashing of reads, or a compressed FM-index representation of the genome. Here, special emphasis was placed on both precision and recall and the performance for different read lengths and numbers of mismatches and indels in a read. Our results clearly showed the significant reduction in memory footprint and runtime provided by FM-index based aligners at a precision and recall comparable to the best hash table based aligners. Furthermore, the recently developed Bowtie 2 alignment algorithm shows a remarkable tolerance to both sequencing errors and indels, thus, essentially making hash-based aligners obsolete. Novel possibilities and challenges have been introduced to genome analysis by the emergence and rapid distribution of next generation sequencing (NGS) technologies. In contrast to traditional Sanger sequencing, the NGS counterpart relies on multiple-fold coverage of each sequenced base by many short sequencing reads. What took the International Human Genome Consortium over a decade and an estimated $300 million to complete, is now feasible within one day’s time and at a fraction of this price. Due to falling sequencing costs, NGS technologies have been extended to many more applications apart from genome sequencing, in particular transcriptome sequencing and quantification.Previously used hybridization-based methods for quantification and characterization of transcripts require careful design of the array platform and knowledge about the transcriptome under investigation. Furthermore, they suffer from cross-hybridization effects and have a limited dynamic range The mapping of sequencing reads to their sequence origin is the first step upon which any subsequent analyses are based. A specific challenge for the sequencing of eukaryotic transcriptomes is the mapping of reads from spliced transcripts to the genome. As read lengths increase, the number of reads spanning exon junctions increases such that alignment to an unspliced reference becomes impractical. Accordingly, a large number of mapping approaches have been developed in recent years to address this problem, including TopHat Due to this importance of read alignment for any application of NGS technologies, many software tools for short read alignment have been published. Although standards such as MAQ et al. published a study comparing accuracy and runtime on genome alignments for increasing genome sizes and eight commonly used alignment algorithms Although newly proposed alignment algorithms are generally compared against a few selected other algorithms, there exist few comparative studies on a wider range of algorithms which could provide some guidance in the choice of the algorithm. Recently, Ruffalo In this study, we address all of these points by performing a comprehensive analysis of 14 algorithms including also read indexing based approaches in the context of RNA-seq experiments. Please note that this is not an evaluation of approaches for assigning reads to their correct position in the transcriptome, but of the underlying alignment algorithms that may be used and, thus, does not focus on splicing detection. For each algorithm, we first determined optimal parameters on smaller training sets for varying read lengths and evaluated stability of the results with parameter changes. On larger test sets, we then evaluated recall and precision as well as runtime and memory requirements. To realistically simulate the conditions observed in RNA-Seq experiments, reads were simulated from transcript sequences. Furthermore, as many of the fastest and best-performing RNA-seq mapping approaches involve transcriptome alignments, simulated reads were aligned to the transcriptome. Our results confirm the superior performance of FM-index based alignment approaches postulated previously and furthermore show that the recently developed representatives of this category have overcome the vulnerability to deviations from the reference sequences due to sequence insertions and deletions.Methods for NGS alignment follow one of two major algorithmic approaches, namely such based on hash tables as known from BLAST and more recent developments based on compressed prefix or suffix array-like structures (FM-index).http://www.novocraft.com), Mosaik , GNUMAP www.seqan.de/projects/razers.html)) are split and stored in a hash table. In contrast to BLAST which compares all positions within a window, NGS aligners enhance sensitivity and robustness by using spaced seeds, i.e., multiple windows in which some positions are allowed to deviate from the reference sequence. As a consequence, multiple seed masks are required to cover various permutations of match and mismatch positions. Several strategies exist for the creation of split seed masks. BFAST uses empirically derived optimal seed masks for given read and genome sizes. MAQ generates an exhaustive list of seed masks to allow for retrieval of any sequence with at most q-gram filtering which gives bounds on the size and number of perfect matches in two strings of given length and total number of edit operations. RMAP takes advantage of this property in order to reduce the number of hash tables and SHRiMP introduces gap-tolerant seeding Algorithms using hash tables build upon quick seeding of alignment candidates which are then extended or discarded using more precise alignment algorithms. In order to quickly find seed locations, either the reference genome . Paired-end reads of 36, 72 and 100 bp with an inner distance (distance between the 3′ ends of the reads) of 250 bp (standard deviation 50 bp) were sampled uniformly from the transcripts using the empirical sequencing error models provided by MetaSim (http://ab.inf.uni-tuebingen.de/software/metasim/). The error model derived from 62 bp Illumina reads was used for the 36 and 72 bp reads, and the error model derived from 80 bp Illumina reads for the 100 bp reads. Default parameters were used otherwise. Novel structural variants were not generated as our study focused on nucleotide-level performance of the evaluated aligners which are not designed to address other tasks than sequence matching. Simulation was based on the transcriptome rather than the genome for a realistic simulation of RNA sequencing experiments and in order to avoid a bias from features that are not generally encountered in RNA-Seq experiments, e.g. repetitive heterochromatin sequences.RNA-Seq reads were simulated based on the ENSEMBL human transcriptome (GRCh37) using dwgsim 0.1.2 from the DNAA package . Alignments of a read to an incorrect location counted as false positives (FP). Reads that could not be aligned to the correct position counted as false negative alignments (FN). If a read was aligned only to wrong positions, it increased the FN count by one and the FP count by the number of its wrong alignments. Thus, the sum of TP and FN yields the number of reads. Since every read had exactly one correct location of origin and should therefore be mappable, there was no measure for true negative alignments. Note that for this purpose, all alignments provided by the algorithm for the specific parameter choice were evaluated. No additional cut-off was applied except filtering parameters provided by the algorithms themselves. Thus, if an alignment algorithm identified multiple locations for a read, the correct alignment was counted as a true positive and all other alignments to distinct genomic locations as false positives. In general, however, the number of false positive alignments for a particular read was at most 1.Precision and recall were calculated using the following equations:We also calculated the When multiple parameter sets for one aligner resulted in equal In addition to alignment quality for the best parameter, the large number of parameter combinations tested allowed us to investigate the impact of parameter changes on alignment performance. Dispersion of the Alignment software was installed and run on the BioQuant computer cluster  with gcc 4.1.2. If multithreading was supported, 8 cores were used. Runtime and memory usage were capped at 10 CPU days and 16 GB, respectively. Total CPU time and memory usage were extracted from PBS job scheduler reports. In particular, for memory usage we extracted the maximum memory used during execution of the job. As the resource usage was only sampled at specific intervals, this value should be considered only as an approximation.Optimal parameter combinations for each of the 14 evaluated aligners and paired-end reads of three different lengths were determined by testing all permutations that appeared to have an impact on alignment quality. The search space was constrained based upon literature research and the respective software documentation. A total of The evaluation of different parameter permutations also allowed us to assess the robustness of the alignment quality to parameter variations. While a large number of parameters (and combinations thereof) promise tunability of the algorithm to a specific problem, the danger of overwhelming the user with complexity should not be underestimated. Ideally, parameter variation should allow the user to trade precision for recall or to alter runtime and memory properties without affecting overall performance. This is exemplified by precision-recall analysis of Bowtie 2 and SOAP2, two aligners with extremely low and high ectively . PrecisiThe best-performing parameter combinations on the training set for each alignment algorithm and read length were applied to the test sets of 5 million paired-end reads each simulated from chromosomes 1-22 (excluding chromosome 21). The results of this large-scale test are shown in Unfortunately, RazerS on the 72 or 100 bp test sets as well as MrFast on all tests sets exceeded the 16 GB memory cap and, thus, could not be evaluated. Similarly, GNUMAP, Mosaik and SHRiMP required more than 10 CPU days and could also not be evaluated. As the latter three algorithms used reference indexing, it is not surprising that the size of the reference constituted the problem and not the size of the read set to be aligned. A test set of the same size, but only sampled from chromosome 1 transcripts and aligned only this chromosome, could be aligned within the time and memory limits by all three algorithms .Although runtime and memory requirements varied widely among hash table based aligners, both hashing strategies were clearly outperformed with regard to memory consumption and runtime by the FM-index based approaches. These generally required less than 1.5G memory and 1 h runtime even for the 100 bp set to obtain a comparable precision and recall. Among these, Bowtie in particular was characterized by very low memory consumption and runtime at a reasonable alignment quality, thus supporting the wide-spread use of Bowtie within many RNA-seq mapping approaches. However, Bowtie’s alignment recall was surpassed by both Bowtie 2 and BWA, in particular for longer reads. In contrast to Bowtie, their recall actually improved with read length, likely due to the fact that they were developed with a particular focus on longer sequencing reads. Here, BWA outperformed Bowtie 2 for the longest 100 bp reads, whereas Bowtie 2 had a much higher recall for the short 36 bp reads. This improved alignment accuracy for long reads compared to Bowtie, however, came at the cost of a significantly increased runtime by a factor of et al.In the previous study by Ruffalo In contrast to simple mismatches, few algorithms tolerate indels by being designed to handle gapped alignments. As a consequence, recall of most algorithms was impaired considerably by indels. Remarkable tolerance to indels, with near-constant performance as indel counts increased, was shown by BFAST, Bowtie 2 and BWA. While the baseline recall of BFAST was already rather low at 0.81, Bowtie 2 and BWA combined this high indel tolerance with very high recall values. In contrast to BWA, Bowtie 2 not only tolerated indels extremely well but also sequencing errors. Among the ultrafast FM-index based alignment algorithms, Bowtie was most vulnerable to indels. Here, already one indel resulted in a reduction of recall to as little as 0.2. This reduced error and indel tolerance likely explains the reduction in overall recall compared to Bowtie 2 and BWA. It also suggests that it is advisable to replace Bowtie by Bowtie 2 or BWA within a mapping approach in case a substantial number of indels is expected and the additional runtime overhead can be tolerated.In this study, we performed a comparison of read alignment algorithms with a particular focus on RNA-seq applications. This is not an evaluation of mapping methods, which also identify spliced reads and have been evaluated elsewhere et al.A major difficulty in comparing the performance of different aligners is to set up a fair comparison in terms of the task and parameters chosen as well as evaluation metrics. In particular, the choice of the parameters can influence the results dramatically. In the recent study by Ruffalo et al.In this way, we found that the largest differences regarding alignment quality were not observed in the precision of the alignments but in recall, in particular with increasing error rates. This fits well with the results of Rufallo The most striking differences we found in this study concerned the memory and runtime requirements of hash table based aligners compared to the FM-index based algorithms Bowtie, Bowtie 2, BWA and SOAP2. The latter algorithms generally performed as good as the best hash-table based aligners on standard tasks with a resource profile that can be provided by a desktop machine. This shows that the unquestioned use of Bowtie by all state-of-the-art mapping approaches is largely justified as even more recently developed hash-based aligners (e.g. SHRiMP) cannot compete with FM-index based ones in runtime or memory requirements.Interestingly, analysis of error profiles demonstrated that hash table based aligners can now be replaced by FM-index based aligners even for applications where a large number of mismatches or indels are expected. This includes, for instance, sequencing of species for which only a closely related genome is known or cancer transcriptomics in which differences from the reference are expected and relevant to pathology. Among FM-index based aligners, BWA and, in particular, Bowtie 2 showed a remarkable robustness to insertions and deletions. Thus, although both create an overhead in memory and runtime compared to Bowie, one of them might nevertheless be a better choice in applications in which a substantial number of indels is expected. In this way, they provide a trade-off between the high tolerance to errors and indels of some hash-based aligners and the dramatically reduced runtime and memory requirements of Bowtie.This analysis also illustrates the importance of evaluating alignment quality compared to the actual number of mismatches and indels in a read and not the average error rate. Our results showed that the reduction of recall due to mismatches and indels is not a gradual process. Instead, as long as the number of differences between read and reference is below a certain amount that can still be tolerated by the algorithm only little changes are observed. However, as soon as this amount is exceeded, recall generally drops dramatically. Furthermore, as only 11% of reads in our analysis had 4 or more base mismatches and only 0.2% of reads contained indels, error- and in particular indel-tolerance had only a small effect on average alignment quality.In summary, this study is relevant for scientists involved in the analysis of RNA-Seq data in several respects. First, it provides a comprehensive evaluation of the performance of state-of-the-art algorithms from all major algorithmic classes used for read alignment. Second, it provides guidance with regard to the choice of algorithm and parameters. Although the task at hand may differ from the situations simulated in this study, the relative performance of the algorithms with regard to alignment quality, runtime and memory consumption as well as their weaknesses and advantages can be extrapolated from this study to other tasks. Third, this study establishes a procedure for identifying optimal parameters using a smaller training set and highlights the importance of evaluating both recall and precision and considering the actual number of mismatches and indels in a read instead of overall error rates. Finally, even if the the alignments algorithms are not used as stand-alone procedures but as integral parts of sophisticated mapping approaches for identifying spliced reads, assessment of their performance is of major importance. As alignment algorithms are generally used in a generic fashion, exchanging the underlying alignment procedure for a better performing one provides one straightforward way to improve the overall performance of a mapping strategy. Here, our study supports to some degree the de-facto standard of using Bowtie as alignment algorithm within RNA-seq mapping pipelines, but also illustrates the potentials of replacing it by other FM-index based algorithms such as BWA and, in particular, Bowtie 2.Table S1Table of optimal parameter sets for each alignment algorithm as determined on a test set of 500,000 reads simulated from transcripts of chromosome 21. The best parameter combination for each aligner was selected based on (PDF)Click here for additional data file.Table S2Evaluation results for alignment algorithms based on reference hashing on a test set simulated from and aligned to chromosome 1 (5 million reads).(PDF)Click here for additional data file.Text S1Details on the algorithms that were examined for this study.(PDF)Click here for additional data file.


The rapid increase in genomic and genome-scale data is resulting in unprecedented levels of discrete sequence data available for phylogenetic analyses. Major analytical impasses exist, however, prior to analyzing these data with existing phylogenetic software. Obstacles include the management of large data sets without standardized naming conventions, identification and filtering of orthologous clusters of proteins or genes, and the assembly of alignments of orthologous sequence data into individual and concatenated super alignments. Here we report the production of an automated pipeline, Hal that produces multiple alignments and trees from genomic data. These alignments can be produced by a choice of four alignment programs and analyzed by a variety of phylogenetic programs. In short, the Hal pipeline connects the programs BLASTP, MCL, user specified alignment programs, GBlocks, ProtTest and user specified phylogenetic programs to produce species trees. The script is available at sourceforge . The results from an example analysis of Kingdom Fungi are briefly discussed. As a direct result of genome biology and high throughput sequencing technologies, the bottleneck in producing multi-gene species trees has shifted from generating data to processing and analyzing data. Genome-based projects stand to contribute ample data for large multi-gene phylogenies, but mining and parsing these data manually is impractical and requires automation. This paper describes a command line program  that brings together a number of bioinformatic applications into an efficient pipeline that inputs unaligned proteins sequences in fasta format and generate species trees from super alignments containing several orthologous protein sequences in a fully automated manner.        Several available methods exist for automated multi-genome ortholog identification with the strategies for finding orthologs roughly divided into two categories, including graph-based and tree construction methods.  Graph-based methods  project we are collecting and analyzing genome-scale data to produce robust phylogenetic hypotheses for deep and problematic nodes within the Kingdom Fungi.  Towards this end we have developed a pipeline of Perl scripts, collectively referred to as Hal, that connect existing programs to automate the following: the mining of conserved orthologous sequences among a designated number of genomes followed by alignment and phylogenetic analysis of these sequences.  The most appropriate model of evolution, for both individual and concatenated super alignments, is determined and sets of concatenated alignments are subjected to a variety of phylogenetic analyses.  Previous and modified versions of this pipeline have already been successfully applied among fungi         The Hal pipeline inputs predicted protein sequences in fasta format from sequenced genomes and produces species trees from multi-gene super alignments.  In doing so, Hal executes the following basic steps: orthologous cluster identification, alignment of clusters, alignment editing, alignment concatenation, model testing, and phylogenetic analysis .  Ortholog identification involves all-vs-all BLASTP, protein clustering, cluster selection and filtering.  If a run is disrupted for any reason, the program can be restarted at the beginning of the last uncompleted step.  Intermediate scripts that process each of the individual steps can also be used independently, provided the expected formatted files are provided.  The main log provides useful information if an error occurred and lists the steps that have been reached.  At the start of a run the following options are available: running the program locally or using a Sun Grid Engine (SGE), choice of alignment program, choice of inflation parameters to use with clustering, minimum alignment length allowed, minimum percentage data allowed per cluster, outgroup designation and choice of phylogenetic program.  All the intermediate files can be in the output or only the final products.  Details about the contents of these files are available in the documentation distributed with the program.  Before sequences are submitted to BLASTALL they are first subjected to a quality check.  Any fasta sequences with duplicated headers or sequences containing non-IUPAC characters are removed.  Currently, there are no standards for the header format among different institutions and sequencing centers, which makes it difficult to automatically parse out a unique name for each fasta sequence.  Instead a map is created of the original headers and a short and unique alias name is provided.        In the following paragraphs each step is discussed along with its default settings, but more details are provided in the program documentation.  Not all options are currently available for modification when Hal is executed, but if started manually some intermediate scripts have more options available for modification to satisfy user preferences.         All-vs-all BLASTP. – The all-vs-all BLASP         MCL clustering, cluster selection and filtering. – A wrapper script takes in the raw BLAST files as input and executes the MCL clustering program         After MCL clustering across a range of parameters, only single copy clusters (one protein per organism) are selected and filtered starting from the most stringent inflation parameter so that only non-redundant clusters remain where all proteins have best hits to proteins from its own cluster.  That is, filtered clusters can only be represented once in the final alignment and they can not contain a protein with a best hit to a protein that resides within another cluster.  With increasing number of genomes compared, however, the number of single copy clusters with all genomes represented decreases quickly.  This phenomenon may be due to gene loss in a particular genome, error in ortholog assessment, poorly or incompletely annotated genomic input data or some combination of the above.  To alleviate this problem and maximize the amount of data available for phylogenetic analyses, an additional option is available to allow clusters that contain missing taxa.  This option allows the user to set the minimum allowed percentage of taxa present per cluster.  For example, setting the minimum included per cluster value at 80% will allow a maximum of 20% missing data within a given cluster.        Alignment programs. – The default alignment program is MUSCLE Alignment Editing. – Currently there are no options given to refine the setting in each alignment program.  Rather, alignment optimization involves a choice between the four alignment algorithms and the ability to edit the alignments using a customized Perl script (remgaps) and Gblocks                 Concatenation of orthologous sequences into super alignments. – Individual alignments that will be used to build the super alignment are filtered by the minimum alignment length allowed as set by the user at the start of the Hal run.  Mapped alias names are exchanged for mapped organism names and all the alignments are normalized so that each alignment has the same taxa in the same order.  Missing data are represented with the question mark character.  Finally, the alignments parsed as described above are concatenated to produce three super alignments , each resulting in different lengths and number of allowed gaps.  Depending on the minimum alignment length value set for the individual cluster alignments, the three super alignments may also differ in the number of represented clusters.         Amino acid substitution model testing. – The program ProtTest is used for estimating the best model for each alignment         Phylogenetic programs. – A choice of four phylogenetic programs is supported in Hal, two maximum likelihood algorithms, PhyML         Because Hal automatically produces three super alignments, three trees are produced by each phylogenetic analysis.  In this pipeline, confidence in tree topology is tested not only by the range of phylogenetic algorithms and bootstrap support, but also by the variation in super alignment construction provided by remgaps, Gblocks conservative and Gblocks liberal.  Additional super alignment constructions may be performed by restarting a Hal analysis with another alignment program specified; in this case, the pipeline will automatically start with alignment of individual clusters .         Hal is a Perl command line program, meaning there is no GUI.  It was developed on a 64-bit Linux architecture, currently running Red Hat Linux 3.2.3, Linux version 2.4.2.1.  However, Hal should be able to be run in any Linux environment.  It is most efficient when using a SGE, which significantly decreases processing time since serial jobs are run on several processors.  Hal can also run on a single machine, but this will take considerably more time depending on the number of taxa and size of the input genomes.  Running Hal on a 32-bit machine may present a problem of insufficient memory for larger analyses.  Included in the Hal distribution is a script, which checks that all the dependencies of Hal are met, helping you to determine what programs are currently on your system.  Hal needs a fairly recent version of Perl, a few standard Perl modules, BioPerl formatdb, BLASTALL, a supported alignment program, Gblocks, a supported phylogenetic program and ProtTest .  More details such as program versions are available in the install document.Drosophila melanogaster as an outgroup.  These genomes represented 8 of the 15 subphyla/phyla currently recognized in the most recent classification of the Fungi  and served as a core dataset for development of Hal.  Analyses were performed across a range of 60, 80 and 100% minimum included taxa per cluster, clusters were aligned using MUSCLE with the most likely model of amino acid substitution assigned to each protein partition of the super alignment, and phylogenetic analyses were conducted in RAxML.  In Figure 2 the topology and bootstrap support of the most likely tree, from an analysis with minimum 80% taxa present and Gblocks liberal alignment, is compared with eight other analyses.  Table 1 provides a summary of cluster statistics for each alignment across all analyses.        Kingdom Fungi currently has more genomes sequenced than any other eukaryote kingdom and the availability of these data is greatly advancing investigations into evolutionary relationships of the Fungal Tree of Life Table 1.  Summary statistics for Hal analyses of 52 fungal genomes. Alignments were produced with and without D. melanogaster included as an outgroup.1Alignments edited using Gblocks liberal settings.2Alignments edited using Gblocks conservative settings.3Alignments edited using RemGaps. 4Most common amino acid substitution model.        These fully automated Hal analyses recovered all currently accepted higher-level phylogenetic relationships  within the Fungi Fig. 2; .  Eight         Significant improvements have been made to the Hal pipeline since its first implementation , resulting in a faster and fully automated pipeline with more user options.  We are continuing with further developments that include:Expansion of user-defined options for alignment and phylogenetic programs.Inclusion of additional ortholog identification methods .Inclusion of taxa for which only transcriptome data  exist.Inclusion of conflict analyses for detection of incongruent phylogenetic signals among orthologous clusters.Inclusion of phylogenetic network algorithms .Inclusion of nucleotide sequence data.        This research was supported by a National Science Foundation grant (DEB-0732993) to J.W.S.  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.        We thank the Center for Genome Research and Biocomputing at Oregon State University for providing computational resources and Chris Sullivan for assistance implementing Hal in SGE.  We also thank the principal investigators, genome institutes/consortia  and the respective funding agencies associated, for the availability of genomic sequences. The authors have declared that no competing interests exist.


Multiple sequence alignment (MSA) plays a central role in nearly all bioinformatics and molecular evolutionary applications. MSA reconstruction is thus one of the most heavily scrutinized bioinformatics fields. Evaluating the quality of MSA reconstruction is often hindered by the lack of good reference MSAs. The use of sequence evolution simulation can provide such reference MSAs. Furthermore, none of the MSA viewing/editing programs currently available allows the user to make direct comparisons between two or more MSAs. Considering the importance of MSA quality in a wide range of research, it is desirable if MSA assessment can be performed more easily.We have developed SuiteMSA, a java-based application that provides unique MSA viewers. Users can directly compare multiple MSAs and evaluate where the MSAs agree (are consistent) or disagree (are inconsistent). Several alignment statistics are provided to assist such comparisons. SuiteMSA also includes a graphical phylogeny editor/viewer as well as a graphical user interface for a sequence evolution simulator that can be used to construct reference MSAs.SuiteMSA provides researchers easy access to a sequence evolution simulator, reference alignments generated by the simulator, and a series of tools to evaluate the performance of the MSA reconstruction programs. It will help us improve the quality of MSAs, often the most important first steps of bioinformatics and other biological research. Multiple sequence alignment (MSA) plays a central role in nearly all bioinformatics and molecular evolutionary applications. Be it to discover sequence structure and motifs or to infer the evolutionary history among sequences (phylogeny), the first step is to compare the sequences by building MSAs. The process of building an MSA is to infer homologous positions between the input sequences and place gaps in the sequence in order to align these homologous positions. These gaps represent evolutionary events of their own. Gaps  are caused by either insertions or deletions of characters (nucleotides or amino acids) on a particular lineage of sequences during the evolution. In this sense, building an MSA is to reconstruct the evolutionary history of the sequences involved.Due to its significant impact on many bioinformatics and molecular evolutionary analyses, MSA reconstruction is one of the most heavily scrutinized bioinformatics fields. Numerous MSA reconstruction methods have been developed . Assessme.g., PREFAB [i.e., providing a fully-resolved MSA. The datasets generated by simulation, with various evolutionary parameter settings, are also useful for evaluating the robustness, consistency, and efficiency of phylogenetic reconstruction based on different MSA methods. The disadvantage of using simulated sequences, however, is that the events during the simulated evolution are limited by the evolutionary models available in the simulators. One must thus choose an appropriate simulator that can mimic the evolutionary history of the gene or protein sequences he/she is interested in.Effective evaluation of MSA methods requires reference alignments. These are the MSAs that are considered to represent the evolutionary history of the sequences most accurately. The majority of currently available benchmark MSA datasets are based on structural alignments of real sequences  with the alignment reconstructed by ClustalW2 v2.1 [The lW2 v2.1 . AlignmelW2 v2.1  is also Pixel Plot allows for a quick comparison between multiple MSAs. As shown in Figure The Simulating members of the lipocalin protein superfamily represents a challenge for many simulators because (i) due to the short length of the lipocalin proteins, each of the 19 subsequences  has a strict length constraint and (ii) all members of the family must contain the conserved motif  nearet al. [• The phylogeny reconstructed by Sánchez et al.  was usedet al. [• The alignment presented in Sánchez et al.  was usedet al.'s alignment using the PROTTEST Web server [• We analyzed Sánchez b server -38 usingb server  with the• We estimated the indel parameters based on the reference alignment and guide tree using the lambda.pl program from the DAWG package . The geoSuiteMSA makes setting up the iSGv2.1 simulation with all these options easily accessible through the GUI. Parameters and support files are organized into four panels based on their intuitive groupings: "Basic parameters", "Advanced parameters", "Edit guide tree file", and "Edit lineage file". Figure The simulation guide tree must be provided in a guide tree file in Newick format. The guide tree file also specifies other parameters including indel parameters. These additional parameters can be set in the "Edit guide tree file" panel Figure . SuiteMSet al. [In the "Edit lineage file" panel, lineages (clades or subtrees) can be added or deleted, and lineage specifications can be edited. In the panel shown in Figure et al.  for the iSG Simulator window as illustrated in Figure Prior to running a simulation, SuiteMSA provides error-checking for potential parameter conflicts. The actual command line used to run iSGv2.1 with all necessary options is shown at the top of the Phylogeny Viewer  in the et al. [As we described before, the performance of MSA methods can be examined against a reference MSA. A reference MSA can be obtained from a benchmark MSA database or by manually-adjusting any MSA relying on our own experience and knowledge on the sequences of our interests. Or we can use a sequence simulator that generates a "true" MSA based on the given evolutionary model. In the previous section, we used the lipocalin superfamily proteins as a case study, and showed how we can simulate members of such a complex protein family. Simulated protein sequences were aligned using different MSA methods. In this section, as a further example, we will briefly discuss how these reconstructed MSAs are compared with the "true" MSA obtained from the simulation as well as the manually adjusted alignment produced by Sánchez et al. .et al.'s alignment of lipocalin proteins is used as the reference alignment (at the top) and compared with the alignment reconstructed by ClustalW2. In this alignment, the area containing the PROSITE motif  is mostly colored blue showing a high degree of consistency. Note that the first five positions of the motif are not consistent (shown in red) due to the gaps inserted in the ClustalW2 MSA. The entire motif region, however, maintains high SPS values. However, the characters in the positions 40 through 44 in the reference alignment are scattered over 17 columns in the ClustalW2 MSA and colored in red, with 0 or very low SPS values. We expanded the comparison and included MSAs reconstructed by MUSCLE and MAFFT using the Pixel Plot. As shown in Figure In Figure Comparison against the true alignment based on simulated lipocalin protein sequences showed the same pattern Figure : a high SuiteMSA provides unique MSA viewers, which allow researchers to quickly identify inconsistencies among MSAs reconstructed by different techniques. It assists in performance evaluation of MSA methods. SuiteMSA also allows users to perform sequence simulation. This further assists comparative analysis of MSAs based on the "true" reference alignment where insertion and deletion events can be mapped individually onto both the guide tree and the true MSA. SuiteMSA's intuitive and user friendly GUI allows for a quick learning curve in using the powerful simulation program iSGv2.1. This provides an opportunity to a wide range of researchers for setting up complex simulation studies quickly and accurately. With the MSA Viewer, MSA Comparator, Pixel Plot, as well as a graphical sequence simulator, the Phylogeny Viewer with graphical editing options, and the Alignment Viewer with indel-event tracking, SuiteMSA contributes a wide variety of unique features to the field of multiple sequence alignment, sequence evolution simulation, and more general bioinformatics research.• Project name: SuiteMSAhttp://bioinfolab.unl.edu/~canderson/SuiteMSA/• Project home page: • Operating system(s): Mac OS X 10.5 or higher, Linux, and Unix• Programming language: java 1.6• Other requirements: iSGv2.1 must be installed per instructions for sequence simulation. ClustalW2 and MUSCLE need to be installed if the user wish to use the GUIs provided with SuiteMSA.• License: none• Any restrictions to use by non-academics: noneCLA implemented SuiteMSA and wrote the manuscript. CLA, CLS, and ENM conceived the project, discussed the results, and revised the manuscript. All authors have read and approved final manuscript.23 lipocalin protein sequences. The 23 lipocalin protein sequences obtained from Sánchez et al. [z et al.  are inclClick here for fileThe reference alignment of 23 lipocalin proteins. The alignment of the 23 lipocalin protein sequences obtained from Sánchez et al. [z et al.  is incluClick here for fileThe phylogenetic tree of 23 lipocalin proteins. The phylogenetic tree of the 23 lipocalin protein sequences obtained from Sánchez et al. [z et al.  is incluClick here for fileSecondary structure predictions for 23 lipocalin proteins. The secondary structures for the 23 lipocalin proteins predicted by PSIPRED are provided in fasta format.Click here for fileThe guide tree file. File name: lipocalinSuperfamily.tree. This file contains the guide tree along with other parameters used for the iSGv2.1 simulation of the lipocalin superfamily proteins. In addition to this file, Additional files 6-8 are used for the simulation. A brief instruction for setting up SuiteMSA using these files and running the simulation as described in the case study is included in the SuiteMSA package distribution.Click here for fileThe simulation root MSA. File name: lipocalinSuperfamily_template.maroot. This file contains the multiple sequence alignment along with the patterns for the motif and template used for the lipocalin superfamily simulation.Click here for fileThe lineage specification file. File name: lipocalinSuperfamily.spec. This file contains the motif specifications for lineages used for the lipocalin superfamily simulation.Click here for fileThe indel-length distribution file. File name: lipocalin.idlen. This file contains the indel-length distribution used for the lipocalin superfamily simulation.Click here for file