The new generation of massively parallel DNA sequencers, combined with the challenge of whole human genome resequencing, result in the need for rapid and accurate alignment of billions of short DNA sequence reads to a large reference genome. Speed is obviously of great importance, but equally important is maintaining alignment accuracy of short reads, in the 25–100 base range, in the presence of errors and true biological variation.We introduce a new algorithm specifically optimized for this task, as well as a freely available implementation, BFAST, which can align data produced by any of current sequencing platforms, allows for user-customizable levels of speed and accuracy, supports paired end data, and provides for efficient parallel and multi-threaded computation on a computer cluster. The new method is based on creating flexible, efficient whole genome indexes to rapidly map reads to candidate alignment locations, with arbitrary multiple independent indexes allowed to achieve robustness against read errors and sequence variants. The final local alignment uses a Smith-Waterman method, with gaps to support the detection of small indels.http://bfast.sourceforge.net.We compare BFAST to a selection of large-scale alignment tools - BLAT, MAQ, SHRiMP, and SOAP - in terms of both speed and accuracy, using simulated and real-world datasets. We show BFAST can achieve substantially greater sensitivity of alignment in the context of errors and true variants, especially insertions and deletions, and minimize false mappings, while maintaining adequate speed compared to other current methods. We show BFAST can align the amount of data needed to fully resequence a human genome, one billion reads, with high sensitivity and accuracy, on a modest computer cluster in less than 24 hours. BFAST is available at  Recently developed massively parallel “next-generation” sequencing technologies have begun to replace the previously dominant Sanger sequencing technology a priori measure the theoretical sensitivity against mismatches and variants before performing alignment based on the underlying choice of genomic indexes.To address the general speed and accuracy limitations, especially in the context of short reads with potential errors, single base variants and insertion/deletions, of the currently described alignment tools, we developed a new algorithm and associated software tool called BFAST (for BLAT-like Fast Accurate Search Tool). BFAST can align giga-scale short read sets with comparable or better speed compared to existing methods, while maintaining higher sensitivity and accuracy for deletions/insertions. While our algorithm is meant to facilitate practical human whole-genome resequencing, it is not restricted to this task and should prove useful for many problems that involve aligning massive numbers of short reads to a reference genome. Here we describe the basic approach and demonstrate BFAST robustness to detect an insertion or deletion, relative to other methods in the real world context of sequence errors and polymorphisms. BFAST also has the ability to be customized for given time or sensitivity requirements, since we are able to BFAST is a powerful and complete means to perform billions of short sequence alignments within the context of large genomes in a highly sensitive and tunable manner. BFAST performs alignment in two steps. First, using multiple indexes of the reference genome, BFAST identifies candidate alignment locations (CALs) for each read. Next, the reads at each CAL are further aligned using gapped local alignment to identify the best match. These processes are supported for direct sequence reads  as well as reads in two base color encoded form, which is the primary output of the ligation-based ABI SOLiD platform. The first step can be substantially tuned for various levels of sensitivity, accuracy, and speed based on the users goals and the error properties of the data. The second step allows for full completion of alignment - i.e. considers all possibilities including SNPs, insertions, deletions, as well as color errors (the latter is relevant for ABI SOLiD data) k), and the total number of ones and zeros as the key width (w). The number of CALs returned depends on the key size as well as the complexity and repetiveness of the target genome, with longer key sizes making the lookup more unique (on average) but generating fewer lookups resulting in reduced sensitivity , indicates that for the non-repetitive part of the genome, a key size of k = 18–22 is sufficient for obtaining a unique lookup the majority of the time, but that a key size of 14 or lower would rarely return a unique CAL (2–107 locations (i.e. Alu elements). Therefore, for the human genome we prefer k = 18 for short reads (40 bp<) and k = 22 for longer reads (≤40 bp) to balance sensitivity and the uniqueness of the lookup. Shorter keys are useful for shorter reads as the number of offsets possible is greater and improves sensitivity. BFAST implements a hash into the index to reduce the lookup time (see j bases (the hash width) of the reference indexes with j≤k. The hash width is always shorter than the key size.The novel contribution of BFAST is the CAL search step, where we tabulate a list of CALs for each read with the goal to include the true (or correct) location within the CALs. BFAST uses multiple indexes of the reference to increase sensitivity of alignment. Each index is a space-efficient suffix array of the reference genome see . An indesitivity . The numique CAL . Converstime see  that conn pieces (this number is chosen for efficiency), although the lookup phase of the alignment takes somewhat longer because each read will undergo a lookup attempt for each partial index. In this mode, BFAST for the human genome can run well on computers with as little as 4Gb of RAM. Since the key width of an index (w) here is less than the read length, we are able to apply the mask at all possible starting positions offset from the beginning of the read to increase the number of lookups performed per read and thus further improve sensitivity. Additionally, to reduce the running time of BFAST at a small reduction in sensitivity, we impose two upper limits during the search. The first requires that a single lookup be ignored if it returns more than K CALs (we use K = 8 for the human genome). The second requires that a read be ignored if the list of CALs for a read grows beyond length M (M = 1280 for the ten indexes with the human genome presented here). The number of CALs can be reduced, which improves performance time but decreases sensitivity. Further discussion of appropriate settings of K and M can be found in the For the human genome and 50 bp reads, we typically use ten indexes with key size 22 with variable key widths, and a hash width of 14. The ten masks for the indexes are chosen using a simple greedy algorithm to maximize sensitivity gain from each additional mask see . The RAMIn the In order to assess the performance of BFAST, we compare BFAST to other available software for the ability to correctly map short reads in the context of different numbers and types of base differences from the consensus human genome such that the use of BFAST for variant discovery in the human genome is apparent and can be reasonably inferred. For this, both simulated and real world datasets are used. In all comparisons, we evaluate the ability of a method to sensitively and accurately find the correct candidate alignment location (CAL) for a given read during the global alignment step. Within BFAST, the CAL is followed by a gapped Smith-Waterman local alignment for each CAL returned, and subsequent assessment of the best alignment to the consensus genome of all possible CALs. In this approach, BFAST sacrifices some speed in the CAL identification process in exchange for completeness and results in gapped local alignment being performed on all CALs. Most other methods do not perform gapped local alignment , or only in exceptional circumstances with paired end data (MAQ). These methods sacrifice completeness for speed, which may be desirable under some scenarios, but will also lessen the ability to identify true variants. We note that in our simulations we do not require that the errors, SNPs, or indels be identified correctly or even found by other methods, but instead require only that the read is placed within 10 bases of the true location. Thus we are only evaluating the methods on their ability to find a correct CAL during the genome-wide global alignment, and not their ability to finish this with correct variant detection. We take this approach for two reasons: first, unlike BFAST, many of the aligners do not perform the full gapped alignment that would be necessary to properly identify indels, and yet we still wish to compare their performance properties to the extent possible. Second, in the context of a resequencing project, variant detection is ultimately a process in which alignment is only the first phase, and which could employ diverse analytical methodologies, such as local reassembly, to return final variant calls. Thus it is most critical to properly position the reads, via a correct CAL, but it is not fundamentally critical for the aligner to fully detect the variant within the read, although it certainly is a useful capacity and obviously can help in finding the approximate location. We also note as well that it is not our intent here to perform a comprehensive comparison of aligners, or to exhaustively compare all available methods to the many possible modes (index choices) of BFAST, but instead to use a representative sample of other reported methods to put the performance of BFAST in context, and to illustrate the key areas where BFAST provides a compelling advantage in the search for variants in the context of the human genome.BFAST was compared under various models of errors and insertion/deletions to other alignment tools to determine if there were observable improvements in alignment in the simulated datasets . We evalWe first consider the relative sensitivity and accuracy of read mapping with variable numbers of mismatches to consensus . The abiTo further highlight particular areas of performance improvement sought in the design of BFAST, we aligned 50-mer reads with randomly inserted variable lengths of deletions  and inseBFAST and some alignment tools have been implemented to align two base encoded reads generated on the ABI SOLiD platform. Since both BWA and MAQ use color quality scores to aid in local alignment, we give simulated color errors a color quality of 20, and 30 otherwise. The mapping of color space reads were evaluated in a different manner as a single true base difference from the genome results in two color errors and a typical machine error results in a single color error. We thus create random read sets with a single SNP and 0–5 color errors  to demonTen base insertions and deletions are most sensitively detected by BFAST as well as demonstrated in In Based largely on the more comprehensive look up strategies and implementation of gapped local alignment, BFAST has one of the slowest running times compared to other methods. This is mainly due to the specific settings used to maximize sensitivity to variants and errors within BFAST, as well as the time spent loading the 10 genomic indexes, which is included in the running time of BFAST. For larger scale datasets such as a whole-genome shotgun resequencing, we are able to partition the billions of reads such that the search and align times dominate relative to the loading times. Thus, BFAST does not scale linearly with these reported times. Although the running times of BFAST are slower in the majority of cases compared to other methods, when considering the need for sensitivity and accuracy of the alignment process in the context of whole-genome resequencing (see simulations above), BFAST has attractive advantages including the ability to be implemented in a parallel computational environment.We have created a new sequence alignment tool that is specifically designed to meet the challenges of practical whole human genome resequencing using short read data (25–100 bases). BFAST has been implemented completely for the handling of large datasets from the ABI SOLiD color space reads as well as direct sequence reads possible with other platforms. The estimated scale of the problem–aligning a billion reads per day on a moderate computer cluster–obviously demands extremely fast alignment algorithms, but more importantly demands high accuracy of the alignments . This is fundamental because the purpose of resequencing is to locate the variants, including insertions and deletions, relative to a reference genome, and to distinguish real genomic variants from inevitable sequencing errors. Thus, it is critical to find the true alignments of reads that do not match perfectly to the reference, so that sequencing errors can be filtered out via multiple coverage, and so that real variants will not go undetected due to systematic under-mapping of reads with the variant. In practice, then, a suitable method must be accurate to meet this challenge and still perform its operations under realistic computer hardware configurations and in reasonable time, and this was the underlying design goal of BFAST.At a practical level, the BFAST performance tests presented here demonstrate various modes of accuracy of alignment for both simulated data and real-world datasets from both the Illumina and ABI SOLiD sequencing platforms. We demonstrate that BFAST is able to substantially increase the number of reads correctly placed in the genome when containing short insertions and deletions and various numbers of errors while maintaining a lower false alignment rate than any other alignment tools. We are able to map sensitively reads that have up to 10% of the bases in the reads being errors, as well as, reads that contain insertions and deletions up to 10 base pairs with modest error rates. This is critical for identifying indels within the genome of interest, and for optimizing independent alignments of paired end reads to identify structural variants. We demonstrate that BFAST is relatively robust to a wide range of errors. Under key conditions, BFAST is improves the likelihood of correct genomic placement of individual reads over a wide variety of scenarios.While BFAST is highly sensitive with the use of 10 indexes, as shown here, if the sensitivity were not satisfactory for a given purpose, to the addition of additional genomic indexes would improve the sensitivity with a modest increase in the computational time. This relative error or variant tolerance of BFAST is highly relevant to two scenarios. First, efforts are being extended to sequence new genomes of species for which a closely related species genomic sequence currently exists. The alignment of the generated short read data must be aligned accurately to use the prior genome scaffold in the presence of unknown base differences. Second, the implementation of new and perhaps more error prone sequencing methods that may generate even larger amounts of data than current methods but with a higher error rate. BFAST will be able to more effectively align these poorer quality reads and render new technologies more useful.The comparison of BFAST to other methods here–MAQ, BWA, Bowtie, SHRiMP, SOAP and BLAT–is not meant to be an exhaustive comparison to all other available programs, nor is it meant to imply that these particular methods cannot provide satisfactory results for large scale resequencing alignment problems. Rather, it is simply to provide a practical context for judging the speed and accuracy of BFAST demonstrate that any method used to align short reads should be assessed for its sensitivity and accuracy in the presence of multiple variants. The examples shown here illustrate that there can be a substantial decreases in accuracy of mapping for certain types of variants, such as small indels, or higher numbers of mismatches, which may be biologically important. Given the complexity of the possible settings, these methods comparison are always incomplete. For instance, we do not evaluate scenarios that would allow the evaluation of certain heuristic choices of the other methods, deferring to the design principle of completeness as motivation for this omission. We note that for many biologically relevant mutation classes that the accuracy of placement by BFAST with 10 genomic indexes is 80–90% thus there is little to be gained by iterative or heuristic processes.Because BFAST uses multiple gapped indexes, BFAST is able to sensitively identify potential alignment positions. Only reads with under 384 CALs are subsequently evaluated by the powerful gapped local alignment algorithm, which distinguishes reads with identical mapping to multiple locations versus reads with a best matched location. By enforcing an upper limit on the number of CALs, BFAST could potentially ignore reads that map to multiple locations but could be placed within the human genome. Thus, in the interests of higher sensitivity alignments, a key area of potential improvement would be to improve the CAL search step with specialized indexes, such that the correct location is within a more limited set of CALs relative to the current implementation of BFAST.Any method that underperforms in sensitivity compared to BFAST would need to reduce the false mapping rate to show improvement over BFAST beyond performance. This new method would need to improve the CAL search step, as the gapped local alignment is optimal, such that for some read the new method includes the correct location in its set of CALs when BFAST does not. This correct location would then be identified by the powerful gapped local alignment. This requires the new method to have greater sensitivity, when the opposite was assumed. As BFAST is tunable for any level of sensitivity, we are able to sensitively map reads as well as controlling for false positives.The choice of alignment tool for each biologist should be tailored to the goals of the sequencing project. While under the specified parameters, the other methods can be somewhat faster for some datasets and slower for others; the fraction of data mapped was typically less than that achieved using BFAST. We note that BFAST does not have default speed or accuracy parameters per se but rather allows the design of index sets with a broad range of speed/accuracy trade offs. The software distribution includes an extensive family of pre-optimized index set masks from which subsets can be selected given any desired accuracy, as specified in the http://bfast.sourceforge.net, and is fully intended for use in production-level whole human genome resequencing efforts, such as are now underway in our own lab. For this reason, we have taken care with the implementation to make it portable, documented, independent of the sequencing platform, and targeted to run efficiently on a cluster. In practice, BFAST can align 1 billion Illumina 55 base-pair reads in 24 hour period on a cluster containing 156 computer computational cores, with suitable amounts of RAM, which makes these alignments practical from a computer hardware point of view. In particular, 20-node cluster with each node consisting of a dual quad core CPU with 24Gb RAM would suffice. We note that the benchmarks reported here ignored multi-threaded computation, and parallel computation, both of which BFAST supports, which in practice proportionately reduces the number of independent nodes required, when multi-CPU nodes and multi-core CPUs are available, as is widely the case. Further, BFAST is implemented in such a fashion that large-scale alignments can be successfully performed in computation clusters where only a fraction (∼20%) of the nodes have the higher RAM needed for efficient indexed lookup (16Gb-24Gb), while the majority of nodes need only have 4Gb of RAM for the local alignments phase of the computations.We offer the BFAST program as a new tool in the available for processing massively parallel sequence data. The BFAST program is freely available at without increasing the overall running time. This will indeed become viable as bigger dynamic memory capacity becomes possible. There is, however, a limit to the strategy of bigger indexes, in that by being resistant to all possible variants by use of a much bigger index, will require longer key sizes in order to limit spurious CALs. For shorter reads, this will begin to reduce the accuracy of any one index, at which point this strategy loses its effectiveness. The optimal parameters will need to be adjusted over time as sequencing technologies produce longer and cleaner reads.From the balanced timing profiles , rather than constructing an artificial random genome to permit the assessment of the sensitivity and accuracy of alignment of short reads that contain variants including errors, single base mismatches, insertions, and deletions as well as combinations. As single base mismatches are an error mode common to all technologies, we investigate high mismatch rates, as well as mismatches in combination with insertions or deletions, as might occur in reads that contain both a real variant and errors, which occurs frequently in practice. We evaluate the different variant states separately in order to obtain accuracy measures for each type of event, as they differ in the degree of alignment difficulty. We examine both the true positive rate, or the sensitivity, to assess what fraction of reads can be located back to their appropriate location, and we assess the mismapping rate, which is determined as the fraction of all reads that are mapped to the genome that are mapped to an incorrect location. Ideally, a method will maximize true positives and minimize mismapping. For these comparisons, we do not require that the exact edits  introduced in a simulated read be observed, but rather that the read be placed approximately in the correct location since some methods can align a read with an indel to the approximate location, but never call or specify the indel since they perform ungapped alignments. We tally the results in this manner so as to perform a more reasonable comparison between the core alignment aspects in the context of the whole human genome of other methods.L letter long substring from the human genome. Each selected string was randomly altered to contain a specified number of mismatches, insertions or deletions, to produce a final read for the variant class of reads.We simulated reads from the human genome by creating sets of reads with a fixed number of variants. To produce a synthetic dataset, we randomly choose an We generated 10,000 reads for each variant class. This number of reads was sufficient to obtain robust performance statistics.In total, 187 different variant-specific nucleotide datasets were created:x mismatches (0≤x≤10).Reads with exactly x letter insertion (1≤x≤10) and y mismatches (0≤y≤5).Reads with one contiguous x letter deletion (1≤x≤10) and y mismatches (0≤y≤5).Reads with one contiguous Similarly, we also generated 10,000 50 bp variant classes in ABI SOLiD color space:x color errors (0≤x≤5).Reads with one SNP and x color errors (0≤x≤5).Reads with one SNP, a 10 bp deletion, and x color errors (0≤x≤5).Reads with one SNP, a 10 bp insertion, and http://bfast.sourceforge.net.The mismatches, and insertion or deletion break points, and color errors (ABI SOLiD data only) were uniformly distributed within the reads. For the nucleotide data, the high number of mismatches, and for the color space data, the high number of color errors, are meant to represent reads confounded by the impact of sequencing errors on both SNP and indel detection. This is especially important for ABI SOLiD color space data due to its higher error rate, which is corrected only after successful genomic alignment. Additionally, the high number of mismatches (for nucleotide data) considered might correspond to a read from a variant dense region, where several single base variants are further confounded by several read errors. Since BWA and MAQ rely on the color qualities for ABI SOLiD data to detect errors, we give color errors a color quality of 20, and 30 otherwise. The simulated datasets are available at We call a read correctly aligned if the read was aligned uniquely within 10 bases of the original location. If two alignments were found with the same best score, the read was not called correctly aligned. In our simulations we do not require that the errors, SNPs, or indels be placed correctly or even found by other methods, but instead require that the read is placed within 10 bases of the true location, thus assessing global alignment rather than local alignment.Section Algorithm Settings for Simulations.We evaluated each algorithm using one compute node, with two dual-core AMD64 processors at 2.0GHz and with 32GB of RAM. Each algorithm was run as a single process, and thus does not take advantage of any multi-threading or parallel processing capabilities of the algorithms, including those of BFAST. This comparison is done only to evaluate the relative speeds of the various algorithms under comparable hardware architecture, since many processors typically would be used in practice, which is the practical solution implemented with BFAST in practice. The precise settings for each algorithm, including BFAST, are described in the For demonstration purposes, we used a 10.9 million 36 base read data set from the human genome. In total, 33 different regions with known mutations across 5 genes were PCR amplified individually and pooled. Amplicon sizes ranged from 191 bp to 762 bp. After purifying each amplicon with QIAGEN PCR Purification Kit, the amplicons were pooled in one tube to create an equal mixture of all products. The sequencing library from the genomic fragments was prepared using manufacturer provided genomic library preparation protocol version 2.3 . Specifically, this dataset consisted of sequence from PCR products known to contain 13 mismatches, as well as 6 small insertions and 6 small deletions, and sequenced at a depth of coverage generally exceeding 1000-fold. We also analyze 3.5 million 55 base paired-end reads of human genomic sequence from our Illumina GAII sequencer. Libraries were generated from genomic DNA. We then selected only the first end of the pair, giving us 3.5 million 55 base pair single-end reads for alignment.One million reads from two different runs of in house generated ABI SOLiD sequencer data was used for all comparisons, as this is a sufficient dataset to offer reasonable comparison. Both datasets consisted of sequences from human genomic DNA, generated by using standard 25 base and 50 base manufacturer supplied protocols.BFAST supports paired-end data by finding CALs for each end separately. Before local alignment, the user has the option to mirror CALs for one end using the other by specifying an estimated paired-end insert length. The paired-end insert length can be inferred by examining paired-end reads for which each end has only one CAL. The option to mirror (or rescue) one end of the read can help either to improve accuracy or to use one end of the pair as anchor for the other. Each CAL for each end is then locally aligned independently. The criteria to choose the best pair of alignments for the ends are then dictated by the user, and can be based on best-combined score, uniqueness, as well as other post-alignment filtering criteria. Further details on paired-end support can be found in the To support ABI SOLiD color space readsSupplemental Materials S1(1.45 MB DOC)Click here for additional data file.


Using default settings, structure alignments generated by several programs  failed to align the respective conserved residues, although LGA reported correct residue-residue (R-R) correspondences when the beta-carbon (Cb) position was used as the point of reference in the alignment calculations. Further tests using variable points of reference indicated that points distal from the beta carbon along a vector connecting the alpha and beta carbons yielded rigid structural alignments in which residues known to be highly conserved in RIPs were reported as corresponding residues in structural comparisons between ricin A chain, abrin-A, and other RIPs. Results suggest that approaches to structure alignment employing alternate point representations corresponding to side chain position may yield structure alignments that are more consistent with observed conservation of functional surface residues than do standard alignment programs, which apply uniform criteria for alignment  as point of reference) along the entirety of the peptide chain. We present the results of tests that suggest the utility of allowing user-specified points of reference in generating alternate structural alignments, and we present a web server for automatically generating such alignments:  Computational methods of protein structure comparison are fundamental to the understanding of protein function and evolution, as well as to applications in medicine and bio-defense. Predictions of potential “druggable” targets on protein surfaces and preferred antigenic regions suitable for diagnostics or therapeutics design, for example, have been derived from computational analyses involving protein structure comparison . SuccessProtein structure comparison programs typically use uniform parameters  positions) along the entire peptide chain and apply them to all residues  and ricin A chain (1br6_A), generated by four sequence alignment programs  and abrin-A (1abr_A) . SequencPairwise global structural alignments were generated for 15 RIPs selected from PDB using LGA with varying points of representation along a Cb-Ca vector, ranging from 0*Cb  to 3*Cb position (-cb:3.0). RIPs were selected from more than 50 PDB structures based on sequence diversity and non-redundancy within the region of interest (corresponding to residues Y91-T116 of 1br6_A). 1j1m_A, which had been solved at very high resolution (1.5 Ǻ), was included as a control for the alignments with the target structure, ricin (1br6_A) .Structure alignment of ricin with other plant and bacterial lectins is known to be problematic in an immunogenic surface region . This reThe D/E/N/Q mis-alignment within the RIP family of proteins is representative of a more general limitation of structure alignment programs. We also revisited a structural alignment performed using the ProSup program to determine whether using LGA with a point of representation at the beta carbon would confirm a putative mis-alignment detected using the beta-carbon post-analysis filter of In order to examine the effects of using alternate points of representation  in aligning protein structures, we modified our local-global alignment (LGA) software  to accepAdditional tests using varying points of representation to align 15 selected RIP structures taken from the PDB were performed to determine how well alternate points of representation faired in aligning the highly conserved D/E/N/Q residues . When a Although the above examples clearly indicate that a Ca-based mis-alignment with respect to chemical (residue) and spatial conservation can be detected and “corrected” by manual inspection of locally applied Cb analysis, this process is by no means simple in the general case. Whereas ProSup’s filter detected the mis-alignment illustrated in It is interesting to note that the variability-based sequence alignment method devised by http://as2ts.llnl.gov/AS2TS/LGA/lga_pdblist_plots.html we provide a service whereby the user may generate alternate alignments based on defined points of comparison representing residue positions along the peptide chain. This service is intended to enable study of specific cases in which sequence- or structure-based alignments using standard methods are suspect in functional regions. It should be stressed that applying alternate points of representation across the entirely of the protein chain in our test producing At


We have developed a phylogeny-aware progressive alignment method that recognizes insertions and deletions as distinct evolutionary events and thus avoids systematic errors created by traditional alignment methods. We now extend this method to simultaneously model regional heterogeneity and evolution. This novel method can be flexibly adapted to alignment of nucleotide or amino acid sequences evolving under processes that vary over genomic regions and, being fully probabilistic, provides an estimate of regional heterogeneity of the evolutionary process along the alignment and a measure of local reliability of the solution. Furthermore, the evolutionary modelling of substitution process permits adjusting the sensitivity and specificity of the alignment and, if high specificity is aimed at, leaving sequences unaligned when their divergence is beyond a meaningful detection of homology. Sequence alignment aims to match homologous characters, nucleotides or amino acids that are descended from a common ancestor. This is complicated by base substitutions that decrease similarity between sequences over evolutionary time and insertions and deletions that add and remove sequence in different evolutionary lineages. From the end user's point of view, the sequence alignment problem is about placing homologous residues in the same alignment columns and positioning gaps to indicate inserted and deleted sequence.Depending on the aim of the analysis, sequences in an alignment can be seen as descendants of an ancestral sequence or a set of sequences sharing a common or a related biological function. Hence, multiple sequence alignment methods have traditionally modelled either the hierarchical relationships among the sequences  outputting pairs of aligned characters. The model of a sequence pair is extended to progressive multiple alignment using a modification of the phylogeny-aware algorithm that distinguishes insertions from deletions  connecting the two levels. Structure classes describe distinct evolutionary processes , and the moves between the classes define a sequence structure (i.e. regions/sites evolving differently); the moves within a structure class describe the character matching process within a given evolutionary process.The model can be seen as a two-level HMM : on the βgh for transitions between structure classes g and h , whereas δh is jointly defined by a structure-specific insertion–deletion rate rh and the evolutionary timex| and |y| are the evolutionary distances from the ancestral node to the two child nodes  to be aligned (a).Probabilities  g and h  are pred aligned a.x and y consisting of characters x1 … xn and y1 … ym. Sequence sites are described with vectors of probabilities, i in sequence x has character a given that the process is in structure class h. If no sequence structure is imposed, the observed character at a terminal node is given a probability of 1 and others are set to 0; if the observed character is ambiguous, the probability is shared among different characters. For sequences with a known structure (e.g. gene annotation), character probabilities for some structure classes can be set positive and for other classes zero. At internal nodes, h.We consider a pairwise alignment of sequences Mh, a conditional probability for a parent character a at ancestral position zk, given the child character distributions, is defined bya and b given |x|, the evolutionary distance between sequence x and its immediate ancestor, and an evolutionary substitution model in structure class h .Character emission is defined by an evolutionary substitution model , and given the evolutionary distances between the two nodes to align, substitution probability matrices that correctly reflect the expected divergence between the sequences are computed (c). Closely related sequences are expected to be similar just by their recent ancestry and, if the base frequencies in the structure classes are not drastically different, the sequences contain little information to distinguish the regions evolving under different processes.The evolutionary modelling of substitution and insertion–deletion processes ensures that the structure classes are correctly scaled for the alignment of sequences that are differently diverged. The character substitution processes are described by instantaneous rate matrices b, and gicomputed c. CloselHowever, correct homology between sequences, especially on nucleotide level that has low information content, may not be detectable even between moderately diverged sequence pairs  we correctly allow for the moves between states Xh and Yh indicating independent insertion–deletion events at the same or neighbouring sites.Given the probabilities for state transitions and character emissions, two sequences are aligned by searching the most probable state path through the HMM. The algorithm finding the path is generally called Viterbi algorithm, and for a pair HMM similar to ours, it resembles the affine gap-cost algorithm of zk as the conditional probabilities of characters a, given the child sites in the pairwise alignment. Given equation xi asyj), in an internal node Mh, Xh or Yh depending on which is the most probable character-matching event.The recursion for pairwise alignment with an affine gap penalty is described in Given all the sites on the alignment path, the ancestral sequence is fully defined and can be aligned with another sequence. Ancestral sequences are technically not treated differently from extant ones.The posterior probability of the process being in a certain state at a given moment is traditionally computed using forward/backward algorithms . We use l, the probability of observing site zk, an ancestor for column k in the alignment, that either matches two sites or creates a gap using structure class h is given byMh, Xh or Yh depending on which is the most probable character-matching event and zk−1 to a matching site zk, and to a site zk that aligns xi against a gap, in structure class h are defined asX, Y or M depending if the previous site was one of the two gaps or a match, respectively (yj against a gap.For sites 1 … ectively . This isfh(z0) and fh(zl+1), respectively. For the backward computation, the initialization and termination terms, bh(zl+1) and bh(z0), simply change their places and then a backward move from a matching site zk+1 to the site zk in structure class h is defined asFor the forward computation, the initialization and termination conditions are defined as in h at the site zk isfE denotes the full probability of the forward recursion, i.e. the sum of all possible paths through the structure classes.Given the forward and backward algorithms, the relative probability of being in structure classes cf. max in Our approach requires normalization of the match and gap scores  across all structure classes h. The score is computed for each transition on the Viterbi path in each pairwise alignment and, if the insertion-aware algorithm . The transition–transversion ratio is set to 2 and character frequencies π in both classes, whereas the instantaneous rate matrices . However, the posterior probability curve is rather smooth and the accuracy of exon prediction based on any cut-off value would be poor. Also, the model describes single unlinked sites and many conserved non-protein-coding regions obtain high probabilities of being aligned by the slowly evolving class (such as 5′ and 3′ UTRs and sequence immediately flanking the exons). With the model codon, which adds three classes describing the periodicity of codon, the separation between the protein-coding exons and the conserved splicing signal becomes clearer, though parts of the 5′ and 3′ UTRs still obtain high probabilities for the codon classes (b). The model detects protein-coding regions purely based on the periodicity of substitution rates and gap lengths of multiples of three, and it may be misled by few random substitutions or gaps that happen to be in the right frame.The simplistic model equences a. Howeve classes b. The moThe performance of our method in the pairwise alignment of human and mouse seems satisfactory but the benefits of structure modelling should be more significant in multiple alignments. First, the alignments of closely related more similar sequences should provide information of the spatial variation of evolutionary processes and help the more difficult alignment of distantly related sequences. Second, multiple sequences provide more information of the sequence structure than two sequences only, and multiple closely related sequences can provide information on features that do not exist in a more distantly related sequence. As the method is progressive, information is generated for each internal node and can be used to study e.g. lineage-specific differences.c (i)).As expected, the alignment of very close sequences, such as human and chimpanzee, does not provide information on the sequence structure and, with the exception of long gaps, the posterior probabilities of different structure classes roughly reflect their background frequencies (not shown). On the other hand, the posterior probabilities for the codon classes in the alignment of five primate species rather accurately match the known protein-coding exons and provide an exon annotation comparable with that of the human–mouse pairwise alignment—with the difference that the former would potentially identify novel exons only existing in primates (figure 3c (ii)) further sharpens the posterior probability curve at the exon boundaries but does not fully resolve the over-prediction of coding sequence in the beginning and end of the gene. Interestingly, the exon seven is consistently predicted to start 50 bases earlier than the true splice site (figure 3c). The upstream region is nearly identical all the way until monodelphis but a one-base insertion in mouse and rat suggests non-protein-coding function (not shown). The inclusion of monodelphis sequence would have only a marginal effect on the exon annotation, and the platypus sequence is incomplete and lacks the first exon.The addition of the rest of the eutherian mammals (figure 3fast/slow and codon, respectively. In multiple alignment, long insertion–deletions and missing data slow down the alignment significantly.Using alignment anchoring, the pairwise alignments of human and mouse sequences took approximately 600 and 1500 s on an AMD Opteron workstation when using the models We have developed an alignment method that allows for incorporating sequence structure information into the alignment process while still taking into account the evolutionary relatedness among the sequences. In contrast to an earlier approach extending the profile alignment , we base


While full genome sequences are still only available for a handful of taxa, large collections of partial gene sequences are available for many more. The alignment of partial gene sequences results in a multiple sequence alignment containing large gaps that are arranged in a staggered pattern. The consequences of this pattern of missing data on the accuracy of phylogenetic analysis are not well understood. We conducted a simulation study to determine the accuracy of phylogenetic trees obtained from gappy alignments using three commonly used phylogenetic reconstruction methods  and studied ways to improve the accuracy of trees obtained from such datasets.We found that the pattern of gappiness in multiple sequence alignments derived from partial gene sequences substantially compromised phylogenetic accuracy even in the absence of alignment error. The decline in accuracy was beyond what would be expected based on the amount of missing data. The decline was particularly dramatic for Neighbor Joining and Maximum Parsimony, where the majority of gappy alignments contained 25% to 40% incorrect quartets. To improve the accuracy of the trees obtained from a gappy multiple sequence alignment, we examined two approaches. In the first approach, alignment masking, potentially problematic columns and input sequences are excluded from from the dataset. Even in the absence of alignment error, masking improved phylogenetic accuracy up to 100-fold. However, masking retained, on average, only 83% of the input sequences. In the second approach, alignment subdivision, the missing data is statistically modelled in order to retain as many sequences as possible in the phylogenetic analysis. Subdivision resulted in more modest improvements to alignment accuracy, but succeeded in including almost all of the input sequences.These results demonstrate that partial gene sequences and gappy multiple sequence alignments can pose a major problem for phylogenetic analysis. The concern will be greatest for high-throughput phylogenomic analyses, in which Neighbor Joining is often the preferred method due to its computational efficiency. Both approaches can be used to increase the accuracy of phylogenetic inference from a gappy alignment. The choice between the two approaches will depend upon how robust the application is to the loss of sequences from the input set, with alignment masking generally giving a much greater improvement in accuracy but at the cost of discarding a larger number of the input sequences. Advances in high-throughput sequencing and computational power have enabled phylogenetic analyses on an unprecedented scale -3. Largeunigene. Since even unigenes rarely cover the full-length of their corresponding transcript sequences, multiple sequence alignments derived from EST unigenes are often gappy. The gaps tend to be clustered at the beginning and/or the end of each unigene sequence, and the positions that are missing often overlap but do not exactly correspond between unigenes  . ESTs ars Figure . TypicalGappy alignments present a number of difficulties for phylogenetic analysis. First, phylogenetic inference is statistically compromised by a lack of data. Second, different regions in an alignment do not necessarily have the same intrinsic substitution rate due to differences in the strength of purifying and positive selection. This may introduce bias if these different regions are more complete in some sequences than in others. One may overestimate the phylogenetic distance between two sequences that overlap in a fast-evolving portion of the alignment and underestimate the phylogenetic distance between two sequences that overlap in only the slowly-evolving regions. Finally, for phylogenetic methods that utilize a pairwise distance matrix, it may not even be possible to compute all pairwise distances due a lack of overlap between some pairs of sequences.The debate about how to deal with missing data in phylogenetic analysis  is not new -23. A nuHere, we use simulated data to specifically examine the effects of EST-like gappiness on phylogenetic accuracy. Our analyses are based on correct alignments, while in real life situations, correct alignments are often unknown and hard to compute, especially in regions that contain gaps. Our alignments therefore represent the "ideal cases" of true alignments. Using these simulated data, we explore the contribution of a variety of factors by comparing alignments that differ with respect to number of sequences, tree topology, alignment length, and gap pattern. We have found that EST-like gappiness results in lower accuracy than the pattern of missing data obtained with incomplete gene sampling in a superalignment, even in the absence of alignment error. To address this problem, we have compared two methods designed to increase the accuracy of phylogenetic inference from a gappy alignment: alignment masking and alignment subdivision. In alignment masking, certain alignment columns and input sequences are excluded from phylogenetic analysis. In real data, alignment masking may also be used to ensure the positional homology of the columns that remain, to eliminate positions that appear to have undergone multiple substitutions, and to exclude systematically misaligned sequences in an automated alignment workflow. Although alignment masking is most often based on subjective criteria, systematic approaches have also been developed -28,51. Bper se, we generated two types of control alignments. For these, the same amount data were removed as for the EST-like gappy alignments, but the residues to be deleted were either concentrated within alignment columns or randomly distributed throughout the alignment .In order to measure phylogenetic accuracy, we used a set of simulated gappy protein alignments that resemble those derived from EST data (see Methods for details). Briefly, 540 full-length protein sequence families varying with respect to their lengths, number of sequences, substitution rate, and tree topology were generated by the Rose software package , and thestQD, which is a measure of how many quartets, or sets of four sequences, differ in topology between two phylogenies [stQD thus ranges from zero  to one (no quartets accurately inferred). Results are shown in Figure stQD had a median of 0 and a mean of 0.0134. For ML, stQD had a median of 0 and a mean of 0.0105. Taken together, these results confirm that the data in the full simulated alignments are sufficient to allow recovery of very nearly true trees, most of the time, using all three phylogenetic inference approaches.We first tested whether the 540 full simulated alignments contained sufficient phylogenetic signal to recover the true trees using Neighbor-Joining (NJ), Maximum Parsimony (MP), and Maximum Likelihood (ML). All comparisons of true trees with estimated trees are given as standardized quartet distances logenies ,32. Our stQD values for the EST-like gappy alignments were substantially higher than for the full alignments, with a median approaching 0.4 in the case of MP. The median stQD for the column-deleted alignments was zero or very close to zero, although the missing columns did lead to some lengthening of the upper tail of the distribution of stQD values. The difference in phylogenetic accuracy was less pronounced between EST-like and random-deleted alignments, and the rank order of performance among NJ, MP, and ML was similar between these treatments. The maximum stQD for MP in the random-deleted alignments was 1.0, meaning that no quartets in the true tree were accurately inferred.We then evaluated the extent to which missing data within these alignments compromise phylogenetic accuracy, and the effect of EST-like gappiness relative to other patterns of missing data. NJ, MP, and ML phylogenies were computed for the 5400 EST-like alignments, the 5400 column-deleted alignments, and the 5400 random-deleted alignments , 82% were the alignments with only 10 sequences, and approximately 75% were generated using gap patterns 7, 8, or 9.REducing Alignments prior to Phylogenetic reconstruction) [One common approach to preprocessing a gappy alignment prior to phylogenetic inference is to mask the alignment such that only columns and sequences containing sufficient and reliable phylogenetic information are included. Here, we used a simple algorithm we have named REAP (ruction) , which istQD of the NJ tree. Because of the prohibitive amount of time that would have been required, MP and ML trees were not calculated. We analyzed the 4901 (90.8%) of the alignments for which phylogenies could be computed for all 36 parameter combinations (i.e. at least four sequences were retained). Evaluation of the computed phylogenies showed that the exclusion of sequences with varying proportions of gaps had much larger effects on phylogenetic accuracy than the exclusion of gappy columns. Analysis of variance demonstrated that all parameters had significant main effects, and a variety of higher-level interactions were significant (results not shown). The parameter with the largest marginal effect on both accuracy and retention was gs, the maximum proportion of gaps allowed in a sequence. Overall, we found a clear trade-off between the accuracy of the phylogeny obtained and the proportion of sequences eliminated from the alignment  were determined to strike the best compromise between topological accuracy and sequence retention, regardless of window size. Thus, unless otherwise noted, we used these parameters with a window size of six for subsequent analyses.To determine the appropriate masking parameters to use in this study, we conducted a factorial experiment in which 36 parameter combinations were tested (see Methods). We recorded the number of sequences retained after masking, and the stQD. For EST-like alignments, masking greatly improved accuracy for all three phylogenetic methods. The greatest improvements were obtained using NJ and ML . MP trees improved less dramatically (the median stQD dropped from 0.3438 to 0.2286). Results were qualitatively similar for random-deleted alignments. By contrast, masking of column-deleted alignments led to less accurate trees using all three phylogenetic methods. For NJ and ML, the masked column-deleted alignments resulted in less accurate trees, on average, than the masked EST-like alignments. This may be due to the nearly complete retention of sequences in the masked column-deleted alignments compared to approximately 80% retention in the masked EST-like alignments  and then computed phylogenies using NJ, MP, and ML  in the simulated alignments  most affected phylogenetic accuracy, we performed analysis of variance on ng Table . This anSubdividing Incomplete Alignments  and is illustrated in Figure In contrast to the approach of alignment masking, in which data are excluded from the analysis, it might be possible to improve phylogenetic accuracy by statistically modeling the missing data. The approach we examined in this study is called "SIA", for In our implementation of SIA, EST-like alignments yielded an average of four subalignments (and a maximum of 16), with a mean subalignment length of 58.9 columns. Column-deleted alignments predictably yielded only one subalignment, while the random-deleted alignments yielded a prohibitively large number of potential subalignments  parameters used to generate the alignments significantly affected the phylogenetic accuracy, we performed ANOVA on stQD for both full and pruned SIA trees. The results generally agreed with those from alignment masking: all simulation parameters had significant main effects and contributed to higher-level interactions , and that alignments with 60 sequences were over-represented in this set. Alignment masking followed by MP resulted in the best trees for ≈9% of the families. Of these, almost 70% were alignments of length 50, nearly 70% contained 10 sequences, and approximately 60% had gap patterns 5 or 7 applied to them. In only ≈7% of the families, the SIA approach resulted in the most accurate phylogeny. Of these, 80% were the shorter alignments, 70% were those to which gap pattern 8 had been applied, and 64% had only 10 sequences. In approximately 20% of all families, no difference was observed across any of the methods. These were in most cases perfect phylogenies (stQD = 0). All of these were alignments of 10 sequences, and masks 9 and 10 were overrepresented among them .For a more detailed comparison of results for the SIA approach and alignment masking followed by either NJ, MP, or ML, we compared ESTs and other partial gene sequences are the predominant source of sequence data for a large and taxonomically diverse set of species. These sequences are tremendously valuable for gene discovery, genome annotation, comparative genomics, marker development, and a variety of other uses ,33. HowestQD for the NJ and ML trees were 0, and the median stQD for the MP trees was 0.004. These results confirm that the presence of missing data itself does not lead to an incorrect phylogeny as long as sufficient data is available for the analysis [Are these missing data really a problem? We found that it was possible to recover accurate trees from alignments in which the missing residues were clustered into columns. Even though half of the simulated alignments had between 50% and 60% missing data, the median analysis ,22,24,25stQD for trees computed from these alignments ranged from 0.17 for ML to 0.34 for MP. When using NJ, the phylogenetic accuracy was even lower for the EST-like gappy alignments than for alignments in which the same number of residues were randomly deleted. One explanation for these results is that for the random-deleted alignments, there is at least some overlap between all the pairs of sequences. For the EST-like gappy alignments, on the other hand, it is common for some pairs of sequences to share no columns in which data are present , and thus no distance can be computed. This poses particular problems for distance methods. For example, PHYLIP reports a distance of "-1.0" for any two sequences that do not overlap in the input alignment. This is taken at face value during execution of the NJ algorithm, leading to a systematic bias toward overly close relationships between sequences in the tree as a result of the lack of overlap between them. The importance of the distribution, and not just the amount, of missing data, was shown earlier in a different context by Wiens [However, EST-like gappy alignments appear to be qualitatively different. When the same amount of missing data was distributed in a pattern typical of EST unigenes, phylogenies were much less accurate: mean by Wiens . In thatstQD of 0.0022 and 0.0026 vs. 0.0 for full alignments). Even for MP tress, alignment masking was able to improve the trees approximately to the level of unmasked NJ trees (stQD of 0.2286). While this may be due solely to the removal of gaps, it may also reflect the removal of alignment positions that have undergone multiple substitutions, thus making the phylogenetic signal clearer in those that remain. Either way, one cannot escape the paradox that the phylogeny is made more accurate by ignoring error-free alignment input. Another important point is that alignment masking comes at the necessary expense of failing to retain all the sequences. On average, 27% of the sequences within an EST-like alignment were excluded by masking in our experiments.We have shown that one can improve phylogenetic accuracy by taking either one of two diametrically opposed approaches. In the first approach, one excludes gappy columns and sequences from the analysis through alignment masking. In our implementation of masking (REAP), we mimic the way it would be performed on real data by also excluding columns and rows that show evidence of misalignment, even though there is no alignment error in our simulation. Most of the trees computed from masked alignments using either NJ or ML methods were comparable to those computed from alignments without any missing data  was able to retain a much higher proportion of the sequences; the median proportion of retained sequences using SIA was 100%. SIA generally, though not universally, led to more accurate trees than those computed directly from the gappy alignment. The greatest improvements in accuracy under SIA were seen in those families that had many subalignments. Where incomplete alignments were divided into 12 or more subalignments, SIA resulted in a more accurate phylogeny in almost all cases. On the other hand, when there were only two subalignments, the phylogeny computed directly from the original alignment was more accurate two-thirds of the time. Perhaps not surprisingly, the number of subalignments was closely associated with the gap pattern used in the simulation. Gap patterns 1, 2, 3, and 8 typically resulted in only one to four subalignments, while gap patterns 6, 9, and 10 typically resulted in a much larger number; thus, certain gap patterns are intrinsically more likely to see an improvement under SIA than others.stQD = 0.002) than when directly computed (stQD = 0.246). The same differential was only about two-fold when using SIA (stQD = 0.127). The phylogenetic accuracy using SIA was thus comparable to the masked MP trees and the unmasked ML trees. Furthermore, the SIA approach is computationally laborious. Taken together, our results suggest that alignment masking is the preferred approach when the distribution of missing data is EST-like in nature.The improvement in phylogenetic accuracy was generally much higher with masking than with subdivision. NJ trees computed from EST-like alignments were over 100-fold more accurate with alignment masking . Our sequence families differed with respect to four factors , or randomly distributed throughout the alignment (random-deleted). For the 5400 column-deleted alignments, columns of characters, i.e., all residues at a given alignment position, were randomly selected without replacement and deleted. In the 5400 random-deleted alignments, the same number of total residues per sequence as in the corresponding gap pattern were independently and randomly selected for deletion, without replacement  and n sequences. Columns are discarded when the proportion of gap characters exceeds a threshold gc, or when the mean sum-of-pairs score within the windowThe alignment masking algorithm REAP works in two steps by first discarding columns and then discarding sequences. Let n matrix . First, sc. Counting only those columns that remain, a sequence i is discarded when the proportion of gap characters exceeds a threshold gs, or for when the mean sum of pairs score to all other sequencesis lower than a threshold ss.is equal to or lower than w = 3, 6, sum-of-pairs threshold scores of sc = 0.5, 1.0, 1.5, column gap thresholds of gc = 0.1, 0.2, 0.5, and sequence gap thresholds of gs = 0.5, 0.7. Because there is no alignment error in the simulated data, the sum-of-pairs score threshold for eliminating misaligned sequences was not varied but instead held constant at a value of ss = 0.25. All 5400 EST-like gappy alignments were masked with each of the 36 different combinations of variable REAP parameters and the resulting masked alignments were input to phylogenetic analyses as described below.To determine the optimal alignment masking parameters, the following REAP parameter values were tested: window sizes of We computed a phylogeny for each replicate using NJ, MP, and ML. Unrooted NJ phylogenies were computed using the PHYLIP programs Protdist (with the JTT substitution matrix) and Neighbor . ML phyloverlap graph is constructed in which each sequence is represented by a node, and the edges between nodes are weighted by the number of shared nongap alignment characters between the two sequences. Using the Bron-Kerbosch algorithm [sc = 0.5, ss = 0.25, gc = 0.1, gs = 0.5 and w = 3 and discarded subalignments with fewer than three columns. In order to identify cliques with common sequence pairs, a subalignment graph is computed in which each subalignment is represented by a node and undirected edges are drawn between subalignments having at least two sequences in common. The connected component that represents the largest number of subalignments is then chosen and the others are discarded. In the absence of any connected components with more than one subalignment, the clique with the largest number of columns is chosen. For subalignments containing more than two columns, pairwise distance matrices are then computed, as above, which results in one or more submatrices.The SIA algorithm takes an alignment as input. An lgorithm , maximalk are scaled by a factor that takes into account (i) the relative rate of substitution for the columns in each submatrix relative to the alignment as a whole and (ii) the relative uncertainty in that estimate as a function of the length of the subalignment. A number of recent papers propose similar methods for combining data from different partitions of a phylogenetic dataset [The submatrices were then combined into one matrix by a linear model in which the distances in each submatrix  dataset ,49. For k based on the average substitution rate of the columns in the subalignment sk and the average substitution rate of all columns in the alignment stHere, we calculated the relative substitution rate for subalignment These rates are known from the ROSE parameters. The scaling factor was then weighted based on the number of columns in a given subalignmentwk is based on that used with real data, and reflects the fact that the precision of the estimate varies with subalignment length. For a subalignment mk, it is computed asThe weighting term wk is that subalignments with larger number of columns are expected to have less variation in their pairwise distance values.The motivation for Pairwise distances that were were not present in any submatrices were imputed using a four-point metric that requires five of the six pairwise distances for four sequences be known and assumes additive tree distances among the sequences ,39. The QD) [QD between two phylogenetic trees is the number of quartets, or sets of four sequences, that differ in topology  between them. Since QD is dependent on the number of possible quartets, and this differs among comparisons, we calculated standardized quartet distances by dividing QD by the total number of possible quartets.We compared the estimated trees with the true trees by measuring the Quartet Distance (QD) ,32 as imQD) . The QD n is the number of sequences in common between the two phylogenies. stQD therefore ranges from zero to one.where Different measures for comparing phylogenies exist, but unfortunately all of them can be inadequate in some cases. We chose the quartet distance for this study because it is appropriate when major rearrangements of individual taxa are expected, as was expected for our data . For exaSH and TJV designed the experiments and analyses and wrote the paper. SH performed the computational experiments and analyses. Both authors read and approved the final manuscript.


The comparison of homologous sequences from different species is an essential approach to reconstruct the evolutionary history of species and of the genes they harbour in their genomes. Several complete mitochondrial and nuclear genomes are now available, increasing the importance of using multiple sequence alignment algorithms in comparative genomics. MtDNA has long been used in phylogenetic analysis and errors in the alignments can lead to errors in the interpretation of evolutionary information. Although a large number of multiple sequence alignment algorithms have been proposed to date, they all deal with linear DNA and cannot handle directly circular DNA. Researchers interested in aligning circular DNA sequences must first rotate them to the "right" place using an essentially manual process, before they can use multiple sequence alignment tools.In this paper we propose an efficient algorithm that identifies the most interesting region to cut circular genomes in order to improve phylogenetic analysis when using standard multiple sequence alignment algorithms. This algorithm identifies the largest chain of non-repeated longest subsequences common to a set of circular mitochondrial DNA sequences. All the sequences are then rotated and made linear for multiple alignment purposes.To evaluate the effectiveness of this new tool, three different sets of mitochondrial DNA sequences were considered. Other tests considering randomly rotated sequences were also performed. The software package Arlequin was used to evaluate the standard genetic measures of the alignments obtained with and without the use of the CSA algorithm with two well known multiple alignment algorithms, the CLUSTALW and the MAVID tools, and also the visualization tool SinicView.The results show that a circularization and rotation pre-processing step significantly improves the efficiency of public available multiple sequence alignment algorithms when used in the alignment of circular DNA sequences. The resulting alignments lead to more realistic phylogenetic comparisons between species. Genomic sequence alignment tools have been playing an important role in comparative genomics and phylogenetic reconstruction. However, traditional sequence alignment algorithms based on dynamic programming are very inefficient when long genome sequences needed to be aligned. To tackle this problem several heuristic based methods have been proposed. The most popular progressive multiple sequence alignment (MSA) method is ClustalW ,2, to whThis limitation is very important since circular DNA sequence alignments are central to a number of biological problems. Every cell has some kind of genome that is circular. Prokaryotic genomes are circular and many bacteria possess extra circular DNA molecules, the plasmids. Eukaryotic cells also contain organelles which possess circular DNA molecules: the mitochondrial DNA (mtDNA) inside mitochondria in all cells; and chloroplast DNA inside chloroplasts in plant cells.Mus musculus, [MtDNA has long been used for phylogenetic analyses. In fact, the absence of recombination in this genome enables an easy and direct inference of the phylogenetic evolution and its fast mutation rate leads to a high discriminative power. Until recently, phylogenetic reconstructions were based on certain regions of the mtDNA molecule, mainly the protein-coding gene cytochrome b when comparing different species  and hypeusculus, ). By theusculus,  and 1,75usculus, . The bliSequencing, the technique employed to determine bases constituting the DNA molecule is performed in fragments, generally overlapping in the ends so that an order can be inferred for constructing the map of the molecule. But the place where a circular genome begins is totally irrelevant and arbitrary. For instance, the first team sequencing the human mtDNA  decided Algorithms for the problem of cyclic sequence alignment have already been proposed in the computer science research field. However, like optimal MSA methods, most existing optimal methods that handle this kind of sequences are very time consuming and seldom used. A simple extension of a general MSA dynamic programming algorithm can be used to compute the edit distance between two cyclic sequences, but requires a quadratic time computation complexity ,17. SeveBased on algorithms closely related to the ideas described above, two software packages have been recently proposed to align circular DNA genome sequences: the Circal package  and the , a very efficient algorithm that finds the best rotation for a set of circular genome sequences that are to be aligned. Firstly, the genomic sequences are circularized. In a second stage the best rotation is calculated based on the largest chain of non-repeated blocks that belongs to all the sequences. These maximum common blocks are obtained with the help of a generalized cyclic suffix tree, which is a new concept introduced in this work. At the end of the process, the users can visualize all the identified common blocks, obtaining a precise idea on how these regions are conserved along the genomic sequences. The new rotated sequences are made available for download and can be submitted to public available MSA tools. At the developed website, several commonly used MSA and visualizations tools are proposed.In this paper, we present the CSA tool The purpose of the proposed algorithm is to find the best rotation among all the possible rotations of each circular genome sequence, in order to improve subsequent multiple sequence genome alignment. Unlike previous algorithms that pursued the same goal, the proposed algorithm is, to the best of our knowledge, the first one that is able to do this task in linear time. This was accomplished by employing the highly efficient suffix tree data structure .In general terms, a suffix tree for a given string is an advanced data structure shaped like an upside down tree that stores all the suffixes of the string and that can be used to efficiently solve many complex string problems. An in-depth explanation of this data structure is outside the scope of this article, but a detailed overview of suffix trees including construction methods and applications can be found in a number of sources (e.g. ). In thicyclic suffix tree. A cyclic suffix tree is a suffix-tree-like structure which represents all the rotations of the sequence . The construction algorithm follows Ukkonen's suffix tree construction method [suffix links and open leaves  sequence. This position-based colouring is more interesting than a length based colouring because it allows, in a much more easy way, the detection of block transpositions among sequences. Before clicking on the alignment image, the table on the right shows the blocks sorted by their length. By left-clicking over a specific section of one of the sequences in the alignment map, the positions table automatically sorts its rows to reflect the correct order of the blocks inside that sequence and it automatically scrolls itself to show the information of the blocks from the selected region/sequence. All the previous described data are available for download at the end of the page.At the CSA tool web page the user can also find a user manual, the algorithm source code and some example sequences.Drosophila melanogaster, the Gallus gallus and the Crocodylus niloticus)  were also conducted. For these sets, the alignment scores and the consensus length obtained when using the ClustalW tool with and without the CSA algorithm were compared.We also tried to compare the results obtained with the CSA tool followed by a linear DNA multiple sequence aligner, with the results obtained by the algorithms available at the Cyclope package . HoweverAfter pre-processing the set of Primate sequences with CSA, 58 homologous chains of blocks with size 8 or bigger were observed, with the biggest one consisting in a block of 16 bp followed by a gap of one base, a second block of 29 bp followed by a gap of 9 bp and a third block of 15 bp, which matches perfectly the tRNA-Met Figure . The thiNote, however, that when more distantly related sequences are considered, the common blocks correspond to single blocks of small sizes (more or less 8 bp) that are still statistical significant if mitochondrial DNA is being considered but not true for DNA sequences of bacterial genomes with million base pairs. In this way, additional care must be taken in the analysis of the sizes of the identified common blocks, when using this algorithmic approach with larger circular genomes from distantly related organisms. In these cases we are dealing with the multiple sequence alignment problem of distantly related genomes that is still an open problem.Some biological properties can help in the evaluation of the alignment results: (1) unique deletion of multiple bases instead of multiple deletions interspersing individual nucleotides; (2) higher ratio transition/transversion, as substitutions between the same type of bases are commonest than between different types; (2) the triplet-constraint in protein-coding genes making it more probable to have deletion in multiples of three than other.As can be observed in Table When comparing different alignment tools, ClustalW adds considerable lower amounts of indels than MAVID. A probable cause for this is that MAVID was especially designed for the alignment of large genomes, while ClustalW is much more conservative and takes longer to achieve results.We tried to investigate where indels were being introduced. This can be visually checked in the tool SinicView, as displayed in Figures When the third set was used so that each sequence was cut each time in different random positions, resulting in 50 test sets of 19 sequences each, the efficiency improvement of CSA was also evident. It was shown that when using the CSA tool on these sets, they all resulted in the same alignment and its score greatly outperformed the scores of each not CSA-processed test set. Figure We have demonstrated that the essential step of circularizing and rotating the mtDNA molecules prior to its alignment can significantly improve the efficiency of current multiple sequence alignment tools, developed for the alignment of linear DNA molecules. This pre-processing step leads to more accurate phylogenetic comparisons between species.To the best of our knowledge the CSA tool is the only web based tool that obtains the best rotation of a set of circular DNA sequences in a very efficient way. The new rotated sequences are made available for further processing and a picture of all conserved block for all the sequences can be found at the result page and can be viewed as a first draft of a future multiple sequence alignment.Future developments of alignment tools should include more real biological mutation constraints, enabling the use of different assumptions in the different parts of the molecules. It is clear that as sequencing strategies advance further, more information will be obtained for complete genomes, which have necessarily a diverse composition. This is clearly the case of the circular molecules of mtDNA and bacterial genomes being rapidly characterized. For instance, non-coding regions could have a less restrictive rate of gap opening; protein-coding genes should incorporate the rule of multiple of three gaps and be less restrictive for substitution at the third-codon position; third-dimension structure can give additional information for the rRNA and tRNA genes alignment.Project name: CSA: Cyclic DNA Sequence AlignerProject home page: Operating system(s): Platform independentProgramming language: COther requirements: noneLicense: NoneAny restrictions to use by non-academics: Free downloads and usage for academics onlyLP and ATF specified the problem and supervised the system implementation. FF implemented the algorithm and coded the application and the web interface. All authors contributed for the writing, read and approved the final manuscript.


Multiple sequence alignments are a fundamental tool for the comparative analysis of proteins and nucleic acids. However, large data sets are no longer manageable for visualization and investigation using the traditional stacked sequence alignment representation.We introduce ProfileGrids that represent a multiple sequence alignment as a matrix color-coded according to the residue frequency occurring at each column position. JProfileGrid is a Java application for computing and analyzing ProfileGrids. A dynamic interaction with the alignment information is achieved by changing the ProfileGrid color scheme, by extracting sequence subsets at selected residues of interest, and by relating alignment information to residue physical properties. Conserved family motifs can be identified by the overlay of similarity plot calculations on a ProfileGrid. Figures suitable for publication can be generated from the saved spreadsheet output of the colored matrices as well as by the export of conservation information for use in the PyMOL molecular visualization program.Bacillus subtilis RecA homolog sequence feature.We demonstrate the utility of ProfileGrids on 300 bacterial homologs of the RecA family – a universally conserved protein involved in DNA recombination and repair. Careful attention was paid to curating the collected RecA sequences since ProfileGrids allow the easy identification of rare residues in an alignment. We relate the RecA alignment sequence conservation to the following three topics: the recently identified DNA binding residues, the unexplored MAW motif, and a unique . Furthermore, we present a curated RecA protein collection that is more diverse than previous data sets; and, therefore, this RecA ProfileGrid is a rich source of information for nanoanatomy analysis.ProfileGrids allow large protein families to be visualized more effectively than the traditional stacked sequence alignment form. This new graphical representation facilitates the determination of the sequence conservation at residue positions of interest, enables the examination of structural patterns by using residue physical properties, and permits the display of rare sequence features within the context of an entire alignment. JProfileGrid is free for non-commercial use and is available from  Comparative nanoanatomy and phylogenetic studies of macromolecules depend upon multiple sequence alignments (MSAs). However, the traditional stacked sequence representation of an alignment proves cumbersome for large numbers of homologs as is prevalent with the proliferation of genome sequences. Early MSA formatting programs facilitated analysis by emphasizing residues with boxes, colors, and shading -3. HowevHere we introduce the JProfileGrid Java software for generating ProfileGrids – a new graphical, tabular representation of alignments. Historically, profiles scored by a distance matrix were used for database searches , althougEscherichia coli because of its crucial protective roles in both recombinational DNA repair  RecA protein sequences both determined by the same group . The ProfileGrid of Figure B. subtilis RecA feature described here is a due to a sequence error. We found the same result in two redundant B. subtilis RecA sequences determined from different research groups . What could be the functional role for these residues? We note that there is controversy regarding the ability of the B. subtilis RecA protein to hydrolyze the cofactor ATP [B. subtilis RecA protein be targeted for site-directed mutagenesis to ascertain if this rare sequence feature influences a potentially unique biochemical activity.The JProfileGrid "highlight sequence" feature can draw attention to any unique residues of a particular sequence within the context of the entire MSA. Here, we analyze the ctor ATP -82. We sProfileGrids serve as a new visual representation of large sequence alignments where the entire information content is presented in a concise form. The JProfileGrid Java software facilitates the creation and analysis of this alignment depiction. With the advent of sequence databases and software programs adopting MSA viewers, the traditional stacked sequence presentation is burdensome for large alignments especially for the interactive analysis of structural patterns and rare features. Thus, we anticipate that the ProfileGrid paradigm will have widespread application in bioinformatics. Finally, we describe and analyze a curated RecA protein data set whose representation as a ProfileGrid will serve as a valuable resource for researchers studying this ubiquitous protein.Project name: JProfileGrid version 1.1.1Project home page: Operating systems: Platform independentProgramming language: Java 1.5 or higherLicense: University of California license; see Any restrictions to use by non-academics: license required for commercial useMSA: Multiple Sequence AlignmentThe authors declare that they have no competing interests.AIR designed the software, collected RecA sequences, performed the bioinformatic analysis & biocuration, and wrote the majority of the manuscript and documentation. AEA collected sequences. ACA wrote Java code and contributed to writing the manuscript and documentation. All authors read and approved the final manuscript and the response to reviewer comments.Multiple sequence alignment of bacterial RecA homologs. A subset of the 300 sequences is shown representing each of the major bacterial phyla. In the alignment, a dash (-) indicates a gap and a period indicates an amino acid identical to the E. coli RecA protein. NCBI Protein database accession numbers are listed at the end unless the data was taken from the TIGR unfinished microbial genomes database. Summary lines above the alignment were calculated from all 300 sequences. The "Bioin" line indicates the bioinformatic structural elements (nanoanatomy) across the entire RecA protein: 12 motifs and the 10 connecting variable regions. "Secon" are the secondary structural elements from the E. coli RecA crystal structure where "a" are α helices, "b" are β strands, "l" are disordered loops, and "?" are disordered termini [E. coli RecA protein sequence. termini . In eachClick here for fileDetailed ProfileGrid of the RecA protein family. The frequency values were calculated from the 300 RecA sequences over the full length (352 residues) of the E. coli RecA homolog (top sequence) that determines the position numbering. The "Major" summary line is the 187 residues conserved above a 70% majority threshold. The 12 RecA family motifs are boxed and labeled  while the connecting variable regions are only labeled. Frequency values are shaded in the ranges of 50 to 69% (light gray), 70 to 89% (dark gray), and 90 to 100% (black). Since we anticipate updating the analysis in the future, this is version 1.0 of the RecA ProfileGrid.Click here for file


We compare the results to direct pairwise alignments, and show that our method gives us higher sensitivity alignments against the target database.Sequence alignment is an important bioinformatics tool for identifying homology, but searching against the full set of available sequences is likely to result in many hits to poorly annotated sequences providing very little information. Consequently, we often want alignments against a specific subset of sequences: for instance, we are looking for sequences from a particular species, sequences that have known 3d-structures, sequences that have a reliable (curated) function annotation, and so on. Although such subset databases are readily available, they only represent a small fraction of all sequences. Thus, the likelihood of finding close homologs for query sequences is smaller, and the alignments will in general have lower scores. This makes it difficult to distinguish hits to homologous sequences from random hits to unrelated sequences. Here, we propose a method that addresses this problem by first aligning query sequences against a large database representing the corpus of known sequences, and then constructing indirect (or  During the last decade, new developments in sequencing technology have resulted in lower costs and larger sequence volumes. Genome and transcriptome sequencing is now performed almost routinely, and the amount of publicly available sequence data has grown enormously. In contrast, correctly identifying the sequence by experimental means remains expensive, and as sequencing volumes continue to grow exponentially the proportion of sequences that can be analyzed in the lab is quickly diminishing. Thus, accurate and efficient computational methods for correct indentification and annotation of newly produced sequences remains an increasingly important challenge.Typically, computational annotation involves searching for similar but known sequences using search tools like BLAST In contrast, curated databases like SwissProt A lack of close homologs tempts the researcher to use lower similarity thresholds, decreasing the reliability of matches and increasing the likelihood of false positives and incorrect annotations. This problem is exacerbated further when the species or process under study is phylogenetically remote from model organisms like human, mouse, or yeast.a) a small, curated database like SwissProt, which would produce precise and detailed descriptions but where low similarity scores would mean that transferred annotations would not necessarily be accurate, or a large database like TrEMBL which would result in higher confidence alignments, but to proteins with less informative and possibly unreliable descriptions.Thus, the dilemma faced by the researcher seeking to identify novel sequences is the choice between indirectly, combining hits against a comprehensive database with alignments between the comprehensive database and the curated one.Here we propose an approach that combines the sensitivity of searching a large database with the detailed annotation from a curated database. This is achieved by calculating alignments against the curated database T-Coffeetriplets of pairwise aligned sequences to increase overall consistency of the multiple alignments by avoiding early local misalignments. These triplets are similar to our transitive alignments, but in contrast to the MSA problem which is NP-complete and therefore must depend on heuristics Our approach is inspired by algorithms for multiple sequence alignment (MSA), and in particular the composition of two pairwise alignments to produce a transitive alignment. We then show how to combine a set of alignments into a consensus alignment, and discuss a scoring scheme and our current implementation.In the following, we first define and describe the alignment of a pair of sequences as a function , satisfying:It follows that the alignment invertible, i.e. there exists a mapping transitive alignment to be the function composition . Given an alignment Let The preceding description ignores some details for the sake of simplicity. In addition to providing a mapping between positions, alignments are also assumed to have an associated score. In order to calculate a score for transitive alignment, we first distribute the alignment score evenly to each pair of positions in the alignment. I.e., given an alignment of score For the entire alignment , the score is calculated as the sum of scores for each coordinate pair, recovering the original score if the alignment is constructed from a single alignment.Given a set of (transitive) alignments We formulate the problem as follows: given a set of scored alignments Let This recurrence is analogous to sequence alignment , with a reward for aligning To explore transitive alignments, we used UniRef 50 Twilight Zone, where pairwise sequence alignments start to produce a considerable number of non-homologous hits The SCOP database We downloaded SCOP 40, version 1.75A. This contains 11 211 proteins from SCOP with at most 40% identity, classified into SCOP's categories of class, fold, superfamily, and family. We used BLASTP to align this set of proteins against itself, and we also generated transitive alignments by using BLASTP to match the SCOP proteins to UniRef 50, and BLASTP alignments from UniRef 50 to SCOP 40, in both cases with an We see that for the same number of false positives, we are able to identify a higher number of true positives using transitive alignments. We also see that as we increase the number of false positives, transitive alignments gain true positives at a higher rate. Using direct BLAST with the default It is also interesting to compare the results for superfamilies and folds. We see that using the looser criterion of folds only marginally increases the number of true positives for BLASTP, but that there is a substantial improvement for the transitive alignments. For instance, at 500 false positives, we find 36 243 true positives using BLAST at the superfamily level, with an additional 459 (1.3%) hits at the folds level. For transitive alignments, we identify 67 932 true positives at the superfamily level, increasing by 6 419 (9.4%) at the folds level.Lepeophtheirus salmonis is a small crustacean that is parasitic on salmonid fish. It is phylogenetically distant from most well-studied species, with copepods believed to have separated from insects more than 300 million years ago The salmon louse ab initio from a draft genome assembly using Augustus As part of the Salmon Louse Genome Project a set of 45 360 putative transcripts have been predicted The input sequences were aligned against UniRef 50 using BLASTX and the UniRef 50 sequences were aligned against SwissProt using BLASTP, in both cases using an In order to compare our approach to a traditional BLAST analysis, we aligned the putative transcripts directly against SwissProt using BLASTX. Using a maximum A direct alignment of the putative transcripts with an entry names) consist of two parts, a mnemonic protein ID (or gene symbol), and a mnemonic species ID. For instance, the protein CYAB_BORPE is the Cyclolysin secretion/processing ATP-binding protein gene (CYAB) found in the species Bordetella pertussis (BORPE). For each transcript, we extracted the gene symbol from the best SwissProt hit, and identified the position (rank) of the highest scoring transitive alignment to a protein with the same gene symbol .SwissProt and UniProt IDs  had no direct BLAST hits to SwissProt. Using transitive alignments, we were able to identify an alignment for 2405 of these. The distribution of alignment scores and lengths are given in SwissProt records are, among other information, annotated with identified protein domains. We therefore compared the transitive alignments to the results of using HMMER with Pfam to identify domains in the sequences. Of the full set of transcripts, HMMER was able to annotate 13 393 (29.5%) with a Pfam domain.  were annotated with a Pfam domain annotation using HMMER, these are shown separately in The most commonly occurring domains in this set, and the corresponding transitive alignments, are listed in An example of a novel annotation is shown in There are now vast amounts of easily available sequence data, and although large resources are committed to curation and experimental validation, the proportion of uncurated data is large and growing. As the raw data becomes cheaper to produce, we need automatic methods that can exploit the raw data directly. The method we present here use uncurated sequence data to provide a context for sequence alignments, and is able use this context to construct alignments with higher sensitivity than direct, pairwise alignments.As a typical transcript annotation experiment, we aligned transcripts to SwissProt, and found that using transitive alignments with a relatively conservative Perhaps the most significant result is the identification of SCOP folds and superfamilies. Here we see that not only does transitive alignment identify the known SCOP relationships more accurately than direct BLAST, but while BLAST mainly find superfamily relationships, transitive alignments are also able to identify many relationships at the more distant fold level. This is perhaps the clearest evidence that our method offers substantial improvement over direct BLAST.Perhaps the best known tool for identifying distantly related homologs is PSI-BLAST Similar to the PSSMs generated by PSI-BLAST, it is possible to identify functionally related proteins through shared motifs or domains, often described using hidden Markov models. Typically, motif or domain databases (e.g. Pfam Although these tools share the goal of identifying homology, they differ from our approach in several important respects. First, although we also make use of multiple sequences to strengthen the results, our aim is to identify individual homologs. Thus, our method serves as a drop-in replacement for regular BLAST, and the method is simpler both to understand and to use as it avoids stochastic models of sequences or sequence fragments. In contrast to PSI-BLAST, our method can be used to annotate nucleotide query sequences directly, and where PSI-BLAST results in a cluster of related sequences, our method results in a set of individual sequences in a specific target database.a priori grouping or classification, and that it produces the alignment of sequence positions, and not just a similarity score.Combining or post-processing pairwise BLAST results is not a new approach, and e.g. taking the best alignment score between groups of sequences has been shown to identify protein family relationships with a higher sensitivity than using stochastic models, at least for small groups of sequences In our analyses we have kept the intermediate and target databases separate. This serves to distinguish the effect of transitive alignments from direct alignments, but will lead to inferior results in some cases when the sequences in the intermediate database match neither query or target sequence well. In practice, this is easily avoided by merging the seqeunces from the target database into the intermediate database.In our current experiment, we have used BLAST-based transitive alignments from nucleotide sequences to proteins, and from proteins to proteins. It is of course possible to use different combinations of sequence databases, or to use different alignment software. It is also possible to extend the chain of databases by introducing more intermediate databases.Similarly, the method is not limited to traditional sequence alignments. For instance, SwissProt contains annotated features for many of its sequences, and one could extract sequence-to-feature “alignments” to identify conserved domains. Functional annotation with Gene Ontology terms Here we have used a simplistic scoring model for an alignment, distributing BLAST scores uniformly over the aligned region, and summing the scores for all aligned position pairs. This scheme has the property that it preserves the original BLAST score, but other schemes are certainly conceivable. The consensus calculation could for instance be modified to use affine gap penalties, and differentiate between global and local alignments. We have not studied transitive alignment scores in detail, and consequently, the alignment score is not interpreted beyond ranking alignments. More work is needed before we can give alignment scores any meaningful interpretation, for instance in terms of The choice of databases to use for sequence alignment is often a compromise between accuracy and precision. By using transitive alignments, we can leverage searches against large databases that give high sensitivity and produce stronger hits against a curated database than one would get directly, and thus make this compromise less severe.The method is conceptually simple, and relies only on standard alignment software and sequence databases. Addition of new sequences to the databases and improvements in sensitivity in the alignment software e.g.,  will thehttp://malde.org/ketil/biohaskell/transalign.We have implemented transitive alignments in Haskell, using version 7.4.1 of the GHC compiler. The source code is available under a GPL license from