Alignment of amino acid sequences by means of dynamic programming is a cornerstone sequence comparison method. The quality of alignments produced by dynamic programming critically depends on the choice of the alignment scoring function. Therefore, for a specific alignment problem one needs a way of selecting the best performing scoring function. This work is focused on the issue of finding optimized protein family- and fold-specific scoring functions for global similarity matrix-based sequence alignment.I utilize a comprehensive set of reference alignments obtained from structural superposition of homologous and analogous proteins to design a quantitative statistical framework for evaluating the performance of alignment scoring functions in global pairwise sequence alignment. This framework is applied to study how existing general-purpose amino acid similarity matrices perform on individual protein families and structural folds, and to compare them to family-specific and fold-specific matrices derived in this work. I describe an adaptive alignment procedure that automatically selects an appropriate similarity matrix and optimized gap penalties based on the properties of the sequences being aligned.http://taurus.crc.albany.edu/fsm.The results of this work indicate that using family-specific similarity matrices significantly improves the quality of the alignment of homologous sequences over the traditional sequence alignment based on a single general-purpose similarity matrix. However, using fold-specific similarity matrices can only marginally improve sequence alignment of proteins that share the same structural fold but do not share a common evolutionary origin. The family-specific matrices derived in this work and the optimized gap penalties are available at  Pairwise alignment of amino acid sequences is a cornerstone sequence comparison method used in a variety of computational applications -4. A mati and j are often observed in the equivalent positions, they have similar biochemical properties and can be substituted one for another in the course of protein evolution. The differences between various similarity matrices are mainly determined by what groups of protein sequences are used to derive them and how the equivalent positions are defined. The mainstream matrices routinely utilized in sequence comparison, such as BLOSUM [In substitution matrix-based alignment, matrix selection is the most important decision the user has to make because once a matrix is selected the values of its elements cannot be easily changed, unlike gap penalties. Most amino acid substitution matrices are similarity matrices based on the same underlying idea: they attempt to account for the similarity between two amino acids by computing how often these amino acids occur in the equivalent sequence positions in related proteins. It is assumed that if two amino acids s BLOSUM  and PAM s BLOSUM , are obts BLOSUM -24. In ts BLOSUM -28. The The advantage of structural superposition is that it allows one to obtain high quality reference sequence alignments for both distantly related homologous proteins and for proteins that share the same structural fold but no common evolutionary origin. Such structure-based reference alignments can be used not only to derive amino acid similarity matrices but also to design benchmarks for assessing the performance of sequence alignment algorithms -33. In aVirtually all existing amino acid similarity matrices are general-purpose matrices, meaning that they were derived by averaging substitution frequencies over many diverse protein families that represent the entire protein universe. General-purpose similarity matrices are required for such task as a sequence database search because in this procedure a query sequence is aligned with millions of diverse database sequences. However, in tasks such as a global sequence alignment only related sequences are typically used and it is often known in advance which protein family these sequences belong to. In such a case, even the best general-purpose matrix may not perform equally well on all protein families, because family-specific substitution patterns were mostly averaged out. For instance, it has been shown that amino acid similarity matrix derived for a specific protein family tends to perform better on proteins from this family than BLOSUM matrices ,35. The In this work, I focus on the issue of finding optimized amino acid similarity matrices for pairwise global sequence alignment. I use a comprehensive set of reference sequence alignments to design a quantitative statistical framework for evaluating the performance of alignment scoring functions on protein family and structural fold levels and apply this framework to study the utility of family- and fold-specific amino acid similarity matrices for global sequence alignment. The results of this work indicate that the quality of pairwise global sequence alignment can be significantly improved by using family-specific similarity matrices.I used the release 1.65 of the Sequence Alignment Benchmark (SABmark)  to derivk, the elements of the group-specific amino acid similarity matrix k Sare calculated as follows:The reference sequence alignments from each SABmark group are used to derive a log-odds amino acid similarity matrix specific for this particular group (referred to as a family-specific matrix for an SUP group and as a fold-specific matrix for a TWI group). The matrices are derived using a modification of the approach suggested by Henikoff and Henikoff . Given ak qand k eare the observed and expected frequencies of amino acid pair  in group k; k fis the total number of amino acid pairs  in group k; k(j) pis the observed frequency of amino acid type j in group k; k nis the total number of amino acid pairs in group k; v is the score for amino acid pair  from the VTML200 matrix [Where 0 matrix . The valk sare rounded to the nearest integer. In order to account for sparse data in small SABmark groups, each element of the final matrix k Sis calculated as the weighted combination of the group-specific score and the general score from the VTML200 matrix. As the total number of aligned amino acid pairs increases, the contribution of the group-specific score increases, whereas the contribution of the VTML200 score decreases. The VTML200 matrix is used in Eq.1 because it is the best performing general-purpose matrix .In the final matrix, all elements i and j, Q, is evaluated by comparing this alignment to the reference SABmark structural alignment for the same pair of sequences, and is quantified by the average of two alignment accuracy measures, D fand M f[The main goal of this work is to compare the performance of various amino acid similarity matrices. This requires a quantitative framework for (1) evaluating the quality of pairwise sequence alignments obtained using a given similarity matrix, and (2) comparing how two different similarity matrices perform when used to align the same set of sequence pairs. The quality of a test alignment between sequences D and fM :I nis the number of residue pairs aligned identically in the test and the reference alignments; R lis the length of the reference alignment; T lis the length of the test alignment. If the value of Q equals 100%, it means that the test alignment and the reference alignment are identical. The test pairwise sequence alignments were obtained by using the Needlman-Wunsch algorithm [m is the gap length, α is the gap initiation penalty, and β is the gap extension penalty. Gap penalties were optimized for each group/matrix as described below.Where lgorithm  with theA and B, in the pairwise alignment of sequences from a SABmark group k is analyzed by means of the two-sided paired t-test. In this approach, the pairs of sequences from group k are aligned using matrix A and the alignment quality score, AQ, is calculated for each pair using Eq.6. Then, the same sequence pairs from group k are aligned using matrix B and the alignment quality score, BQ, is calculated for each pair. After that, the distributions of alignment quality scores A Qand B Qare compared using the two-sided paired t-test. The outcome of the t-test is considered to be statistically significant if its p-value is less than 0.05. To avoid over-fitting, the alignment quality scores are obtained using a 3-fold cross-validation. In this procedure, each SABmark group is randomly partitioned into three non-overlapping sub-sets, with two sub-sets used for training and the remaining sub-set used for testing. The process is repeated three times, so that each sub-set is used for testing once. In the case of a general-purpose similarity matrix, "training" means optimizing gap penalties for this matrix. In the case of a similarity matrix specific for a SABmark group k, "training" means deriving the group-specific matrix itself and optimizing gap penalties for this matrix. "Testing" means using the matrix and optimized gap penalties obtained during the training step to align sequences from the test sub-set. During cross-validation, gap initiation, α, and gap extension, β, penalties for a given group k and a given similarity matrix A are optimized using the following grid search procedure: Sequences from the training set of group k are aligned using matrix A, and all possible combinations of integer gap penalties in range 1≤α≤50, 1≤β≤30 are tested; the combination  that results in the highest average quality score is selected as the best and is used with matrix A to align sequences from the test set of group k.The performance of two given amino acid similarity matrices, In this section I compare the performance of existing general-purpose amino acid similarity matrices using the 3-fold cross-validation evaluation protocol described in the 'Matrix evaluation procedure' section. For this purpose, I chose two most popular series of matrices derived from multiple sequence alignments of evolutionary related sequences, the PAM matrices  and the The results reported in Tables In this section test the performance of the group-specific approach. In this test, a group-specific similarity matrix is derived for each SABmark group according to Eq.1 and compared to the general-purpose matrices using the 3-fold cross-validation evaluation protocol described in the 'Matrix evaluation procedure' section. Only groups that have 10 or more reference alignments were used in this test because in smaller groups the total number of observed amino acid exchanges is not sufficient to derive a meaningful group-specific matrix. The results of this comparison on the SUP and TWI sub-sets are shown in Table An important question is how similarities between aligned sequences affect the performance of group-specific matrices. Do these matrices tend to perform better than general-purpose matrices on groups that contain many similar or many dissimilar sequences? In order to answer this question, I calculated two parameters for each SABmark group: (1) the average difference in the quality score (Eq.6) between alignments obtained with the group-specific matrix and with the VTML200 matrix, and (2) the average sequence identity. The scatter plots of these parameters for the SUP and TWI sub-sets are shown in Figures D1:Finally, I studied how the family-specific matrices differ from the original VTML200 matrix, and how the family-specific matrices that perform better than VTML200 differ from the family-specific matrices that do not. The fold-specific matrices were not studied in detail because they do not appear to improve alignment quality significantly. First, the family-specific matrices were compared to the original VTML200 matrix by using the average difference between matrix elements, k sis the score for amino acid pair  from the family-specific matrix k; v is the score for amino acid pair  from the VTML200 matrix; N is the total number of family-specific matrices. The scatter plot of the average distance D1 vs. the original VTML score v for all 210 amino acid pairs is shown in Figure etc.) or similar  amino acid pairs that receive high scores in VTML200 tend to receive lower, but still positive, scores in the family-specific matrices. On the other hand, dissimilar amino acid pairs  that receive low scores in VTML200 tend to receive higher, but still negative, scores in the family-specific matrices. The higher the magnitude of the original VTML200 score, the more it tends to be affected in the family-specific matrices.Where D2:Second, the family-specific matrices that perform significantly better than VTML200 were compared to the family-specific matrices that perform similarly to VTML200 by using the difference between the average matrix elements, k SBis the score for amino acid pair  from the kth family-specific matrix that performs better than VTML200; k SSis the score from the kth family-specific matrix that performs similarly to VTML200; NB is the number of matrices that perform better than VTML200; NS is the number of matrices that perform similarly to VTML200. The family-specific matrices that perform significantly worse than VTML200 were not used because there were only 5 of them. To remove the cases that show a trend toward statistical significance, matrices that perform similarly to VTML200 were defined as the ones that received the p-value greater than 0.1 during the matrix evaluation procedure .Where BP and D2 are shown in Figures etc.), while assigning higher scores to many dissimilar pairs (such as hydrophobic-hydrophilic pairs)  Figure . Notablyk, then the family-specific matrix k and the optimized gap penalties for this matrix are used to align these sequences. If the sequences are assigned to different families or if at least one sequence is not assigned to any family because of the absence of significant hits, then VTML200  with the gap initiation penalty of -15 and the gap extension penalty of -1 is used to align these sequences. These default gap penalties were determined by the grid search procedure as the best-performing general-purpose combination for VTML200 - among all combinations tested,  results in the highest average alignment quality score calculated for the dataset that contains all SUP groups pooled together.The analysis of the performance of the group-specific matrices described above is based on the knowledge of the group membership for each sequence used for testing, meaning that a correct group-specific matrix is always selected for each alignment by default. However, in the real world we do not always know in advance what particular family a given pair of homologous sequences belongs to. How to select an appropriate family-specific matrix in this case? I used BLAST  sequenceThe results of the assessment of the performance of the adaptive alignment procedure on the SUP sub-set, as determined by the 3-fold cross-validation, are shown in Table 1. I utilized a large set of reference alignments obtained from the structural superposition of homologous and analogous proteins to design a quantitative statistical framework for the comparative evaluation of the performance of alignment scoring functions in the pairwise global sequence alignment. This framework was applied to study how the existing general-purpose amino acid similarity matrices perform on individual protein families and structural folds and to compare them to the family-specific and fold-specific similarity matrices derived in this work.2. Among all the general-purpose matrices tested, VTML200 is the best-performing one. It produces global pair-wise sequence alignments most similar to the reference alignments derived from structural superposition. However, the results of this work suggest that the performance of each general-purpose matrix may be significantly biased towards a certain set of protein families. Even the top-performing general-purpose matrix cannot universally outperform other matrices on all protein families and all folds.3. Using protein family-specific similarity matrices and optimized gap penalties can significantly improve the quality of alignment of homologous sequences compared to the traditional sequence alignment based on a single general-purpose similarity matrix. However, using fold-specific matrices can only marginally improve sequence alignment of proteins that share the same structural fold but do not share a well-defined common evolutionary origin.4. I presented an adaptive alignment procedure that automatically selects an appropriate amino acid similarity matrix and optimized gap penalties based on the properties of the sequences being aligned. This procedure does not require a manual assignment of evolutionary relationships and can be used for an automated optimized alignment of homologous protein sequences in the absence of a prior knowledge of what protein family these sequences belong to. The current limitation of the presented adaptive alignment procedure is that it is directly applicable only to single-domain proteins or to individual domains from multi-domain proteins. This limitation will be addressed in the future.http://taurus.crc.albany.edu/fsm.The protein family-specific amino acid similarity matrices derived in this work and the associated optimized gap penalties, along with the optimized family-specific gap penalties for the VTML200 and BLOSUM50 matrices, are available on-line at The author declares that they have no competing interests.IBK conceived of the study, performed it, and drafted the manuscript. All authors read and approved the final manuscript.


Traditional genome alignment methods consider sequence alignment as a variation of the string edit distance problem, and perform alignment by matching characters of the two sequences. They are often computationally expensive and unable to deal with low information regions. Furthermore, they lack a well-principled objective function to measure the performance of sets of parameters. Since genomic sequences carry genetic information, this article proposes that the information content of each nucleotide in a position should be considered in sequence alignment. An information-theoretic approach for pairwise genome local alignment, namely XMAligner, is presented. Instead of comparing sequences at the character level, XMAligner considers a pair of nucleotides from two sequences to be related if their mutual information in context is significant. The information content of nucleotides in sequences is measured by a lossless compression technique.Experiments on both simulated data and real data show that XMAligner is superior to conventional methods especially on distantly related sequences and statistically biased data. XMAligner can align sequences of eukaryote genome size with only a modest hardware requirement. Importantly, the method has an objective function which can obviate the need to choose parameter values for high quality alignment. The alignment results from XMAligner can be integrated into a visualisation tool for viewing purpose.The information-theoretic approach for sequence alignment is shown to overcome the mentioned problems of conventional character matching alignment methods. The article shows that, as genomic sequences are meant to carry information, considering the information content of nucleotides is helpful for genomic sequence alignment.ftp://ftp.infotech.monash.edu.au/software/DNAcompress-XM/XMAligner/.Downloadable binaries, documentation and data can be found at  Advances in sequencing technology allow high throughput production of biological sequences in sequencing laboratories around the world. The exponential increase of genomic data extracted recently introduces the need for analysis techniques that can handle the large amount of data. This is very challenging as conventional analysis methods can be overwhelmed by volume and misled by statistical biases. It is important to develop novel tools that are time efficient and able to cope with the diversity of the data.global alignment and local alignment. Global alignment attempts to match entire sequences from end to end and thus is suitable for comparing short sequences that are expected to have similar structures and functions such as proteins or genes. On the other hand, local alignment searches for conserved regions, possibly reordered, between two sequences. Local alignment is thus more suitable for analysing long sequences, such as chromosomes or genomes, especially from distantly related species where significant insertions, deletions and large rearrangements may have occurred.One of the most important tools for sequence analysis, if not the most important one, is sequence alignment which attempts to arrange biological sequences to identify regions of similarity. Similarities between sequences can provide clues to discover the evolutionary relationship between species, to annotate new sequences and to compare an un-known sequence against existing sequences in a large database. There are two broad kinds of sequence alignment, namely seeds of k consecutive exact matches. Seeds are then extended, by limited dynamic programming, to allow for mutations and gaps.Most existing alignment methods are inspired by the dynamic programming approach ,2 which Since 1995 when the first genome of a free-living organism was sequenced , Sim4 77, SSAHA Most traditional alignment methods rely heavily on a scoring scheme that includes a substitution matrix, which describes the mutation rates between nucleotides or amino acids, and other parameters such as gap penalties. However, these methods lack a well-principled objective function to measure the performance of a set of parameters: "There is considerable disagreement among biologists about the 'right' choice of parameters" . Using ainformation, a new alignment methodology that performs alignment based on the information content at each nucleotide position is proposed here. The methodology is based on information theory An evaluation of an alignment tool compares the homologues predicted by the tool against "true" homologues. True homologues in genomes, however, are not always reliable as they are often located by automated tools or by subjective prediction by human experts. Some alignment benchmarks based on real data such as BAliBASE  and JareDrosophila genomes. We selected ten alignments at random from their 10000 alignments. Each alignment contains homologous sequences that were generated based on homologous non-coding regions of five species Drosophila melanogaster, Drosophila simulans, Drosophila yakuba, Drosophila ananassae and Drosophila mojavensis. Each sequence is 1000 bases long. We generated five unrelated simulated genomes of length 100 kb, and inserted the ten homologous sequences of each species into a simulated genome at random positions. The generation resulted in five simulated genomes, each of which contains ten homologous regions.We first experimented using simulated data. We generated our artificial genome benchmark data set in which homologous regions are scattered around the genomes in a random order. These homologous regions were taken from the alignment benchmark in  for whicD. melanogaster homologous sequences against each of the other four genomes. The object of the alignment was to locate the homologous sites from each genome. Sites resulting from insertions were not considered homologous. The data set consists of four pairs of simulated genomes, namely, D. melanogaster - D. simulans, D. melanogaster - D. yakuba, D. melanogaster - D. ananassae and D. melanogaster - D. mojavensis in order of increasing genetic distance. In order to investigate different statistical distributions, we generated two sets with different statistical properties from these four pairs. In the first set, unrelated regions were generated from a uniform distribution (uniform set). In the second set, unrelated regions were generated from a statistically biased distribution in which the frequencies of A, C, G and T are 40%, 10%, 10% and 40% respectively (biased set). In total, our benchmark contained eight pairs of simulated genomes.We performed local pairwise alignment of the simulated genome containing score cut-off (co) and set the word length (wl) to 10; for Dialign, we varied the threshold (thr); for Nucmer, we varied min cluster (c) and set minmatch (l) to 14 and maxgap (g) to 120; for Blastz, we varied scoring threshold (hspthresh); for XMAligner, we varied homology ratio threshold (r) and set the hash key size to 10. All other parameters were set to their default values for DNA alignment.The programs XMAligner, Dialign , Chaos , Nucmer D. melanogaster - D. ananassae and D. melanogaster - D. mojavensis. On the biased set, XMAligner outperformed both Blastz and Nucmer, especially on distantly related sequences. In such biased data, spurious matches occur more often than in more uniformly distributed data. As a result, Blastz, Chaos and Nucmer were misled by the bias of the data. On the other hand, XMAligner examines the information content of every nucleotide. In a low information region, the information content of a non-homologous nucleotide is calculated accordingly and thus spurious matches reported are minimised.Figure D. melanogaster genome in each pair. For each set of parameters, we varied the homology ratio threshold to obtain different sensitivity and specificity values. The ROC curve for each set of model parameters is displayed in Figure We performed an experiment to verify the proposition that the best alignment of two sequences leads to the best compression of a sequence on the background of the other. The experiment was performed on the four biased genome pairs. We first varied the parameters of the compression model, namely the hash table key size, the context length and the expert panel limit, so that different compression results could be obtained. The compression performance of each set of parameters is measured by the average compression of the simulated We also performed experiments on real data. We used the Jareborg data set  which comin cluster (c) and set minmatch (l) to 6 and maxgap to 30; for Sim4, we varied HSP threshold (C) and set word size (W) to 10. The sensitivity versus specificity ROC curves for these algorithms are plotted in Figure For a pair from the data set, we applied each algorithm to align the mouse sequence against the human sequence, and compared the HSPs detected in the mouse sequence to the annotated mouse exons. The parameters for XMAligner, Chaos, Dialign, Nucmer and Blastz were the same as in the previous experiment. For Promer, we varied Plasmodium species, namely P. falciparum, P. knowlesi, P. vivax, P. gallinaceum and P. yoelii. The genome sequences and their annotations were obtained from PlasmoDB release 6.2 [P. falciparum and P. vivax are malaria parasites on human while P. knowlesi and P. yoelii cause malaria in monkey and rodent respectively. P. gallinaceum is a bird malaria parasite. The nucleotide compositions in these genomes are very different. The AT content in the genome of P. falciparum is as high as 80% genome-wide, even 90% in introns and intergenic regions, while the AT content in the P. vivax genome is just 57.60%.We used XMAligner to align the genomes of five ease 6.2 . Of the Plasmodium species exhibit an extremely difficult example of sequence alignment. The highly skewed distributions of genomes of species such as P. falciparum, especially in non-coding regions, may lead to the return of spurious matches. Furthermore, in different stages of their life-cycle, Plasmodium species interact with the mosquito vector and the vertebrate host. The strong evolutionary pressure from these interactions has resulted in different codon preferences among the genomes of Plasmodium species. Indeed, the AT content of coding regions of P. falciparum is as high as 76% while the AT content of coding regions of another human malaria parasite, P. vivax is only 53%, although the two species have similar metabolic pathways and their proteins share a high level of identity [The genomes of identity .P. falciparum and P. knowlesi genomes against each of four other genomes and against the concatenation of these four genomes. The similar regions detected during alignment were compared with the exon annotation. We compared XMAligner with Blastz [scoring threshold (hspthresh) of Blastz, minimum cluster (c) of Nucmer and Promer, and homology ratio threshold (r) of XMAligner to get several different values of sensitivity. Other options are presented in Table We aligned each of the h Blastz , Promer h Blastz , which aThe alignment of one genome against another by XMAligner took about 40 minutes. To get high sensitivity, we performed alignment in both forward and reverse directions, and then combined both alignments. The total time for alignment of a pair of sequences therefore was about 80 minutes. The running time of Promer was shorter, about 4 to 5 minutes for alignment one genome against another, and 20 minutes to align one genome against the four other genomes. Nucmer is even faster, it needed only one minute for pairwise alignment and four minutes for aligning one against four genomes.P. falciparum and P. knowlesi are shown in Table The sensitivity and specificity of exon detection of the three programs on the genomes of P. knowlesi genome against P. vivax, these being closely related. In the alignment of distantly related genomes, Nucmer obtained a sensitivity of no more than 20% in most cases. Promer performed significantly better than Nucmer on the data, although the matching techniques of the two algorithms are similar, except that Promer performs alignment at the protein level while Nucmer aligns at the nucleotide level. Blastz performed better than Nucmer, but was inferior to Promer on aligning these sequences.Nucmer performed poorly on most cases, with the exception of aligning the P. knowlesi and P. vivax, where XMAligner was more sensitive but less specific. With such a close relationship, many regions other than exons also tend to be conserved. While Promer translates DNA to proteins for alignment, the annotation of just codons is clearly advantageous to Promer's specificity.Although XMAligner aligns sequences at the nucleotide level , it showed a much higher level of both sensitivity and specificity than Promer in the alignment of most pairs. The only exception is the closely related pair X against a sequence Y , XMAligner outputs the sequence of information content of X and the sequence of the conditional content of X given Y , along with a list of HSPs. The toolkit can read these information sequences, manipulate and display them. The annotation of the sequences can also be visualised by the toolkit.We have incorporated the output of XMAligner into InfoV toolkit  for visuP. vivax and P. falciparum genomes from PlasmoDB version 5.4. We applied XMAligner to align contig ctg6843 from the P. vivax genome against the genomes of P. falciparum. The information content sequence of the contig and the conditional information content sequence of the contig given the P. falciparum genome were generated by XMAligner. The information content sequences were loaded into InfoV for viewing. The visualisation of these information sequences and the alignment is shown in Figure In an earlier publication , we perfInfoV is able to display the annotations of a sequence and the HSPs from an alignment. The two rows of red and blue boxes near the bottom of the viewer in Figure P. falciparum. These regions showed a high level of similarity but was not annotated in PlasmoDB 5.4 version. The cluster of these region starts at position 491038 in the ctg6843, and is about 15000 bases long. Its counterpart from the P. falciparum genome starts at position 6971447. We tracked down and found that this area in the P. falciparum genome is a cluster of three genes MAL7P1.203, MAL7P1.320 and MAL7P1.204. The information of the alignment of an HSP is shown in Figure PVX 081792 in the P. vivax genome.During our experiment, we noticed a cluster of HSPs which paired regions in contig ctg6843 to some annotated coding regions in the genome of Most genomic alignment methods have four major components: (i) an indexing technique for locating seeds, (ii) a method for extending seeds, (iii) a method for assigning score to each local alignment, and (iv) a method to evaluate the significance of an alignment. XMAligner presents novel technique for (ii), (iii) and (iv) while it can use any existing methods for (i) from conventional alignment approaches for to propose align experts. Indeed, XMAligner provides option to use hash tables, gapped hash table, suffix trees and suffix arrays, on the standard alphabet  or on the {purine, pyrimidine} alphabet. Other techniques will be implemented in the near future. Most importantly, the suitability of each seeding technique can be measured by the compression objective function.With reference to the traditional dynamic programming approach, an align expert proceeds diagonally. This is similar to gap-free extending seeds. However, there can be more than one align expert employed at any time. If there are gaps in a homologous region, some neighbouring expert(s) would be proposed. Though each align expert can suggest a gap-free HSP, the panel of experts in XMAligner can handle gaps implicitly. This also allows XMAligner not to make any assumptions about gap scores.The matching scores in the traditional dynamic programming approach are calculated based on an information theory perspective . Indeed,Equation 11 shows that the mutual information of an HSP is in fact the traditional alignment score of the HSP which is also measured by the logarithm of the odds ratio of the probability that two symbols are related and the probability that they are independent. However, XMAligner adaptively estimates these probabilities based on the context of the pair of symbols. For example, in a low information region, the information content of a more frequent symbol is lower and its alignment score is computed accordingly. Unlike the "pairwise statistical significance" approach in  which loXMAligner might find multiple segments in the reference sequence that are strongly related to a similar segment in the query sequence. The degree of relatedness is specified by the conditional information content of the segment given each related segment on the reference sequence. This can be used as a ranking to guide further investigation of such an identified segment.Most existing alignment algorithms lack an objective function to indicate which parameters are the most suitable for the data. Objective functions are very important for applications like sequence alignment because biological data are so diverse. It is very hard to anticipate which parameter values capture the essence of the data and will give the best results, especially for data that are not well studied. The objective function provided by XMAligner naturally guides parameter estimation and improves alignment quality.This article presents XMAligner, a novel sequence alignment approach that matches long sequences at the information content level. It considers the information content of the nucleotide at each position during the alignment process. The information content is determined by examining the context of the nucleotide. Unlike traditional alignment algorithms, XMAligner reports aligned regions from two sequences if there is significant shared information between the two regions. The approach is shown to outperform the conventional character-matching approaches, especially for distantly related sequences and sequences with statistically biased composition. The method is able to align eukaryote genomes with only a modest hardware requirement. The output from XMAligner can be integrated into a visualisation tool to aid the analysis of sequences.We argue that, since genomic sequences are meant to carry information, aligning in terms of information content is a better approach for genomic sequence alignment. Each nucleotide should be examined within its context. The approach is better suited than the conventional approaches which measure the alignment score of matching symbols entirely based on a fixed scoring scheme.MDC developed methods, performed experiments, analysed data and wrote the paper. MDC, TID and LA contributed to the mathematics. TID and LA supervised the work, and participated in discussions on algorithms, biology and statistics, and in the writing of the paper. All the authors read, edited and approved the final manuscript.


Multiple sequence alignment is a crucial task in a number of biological analyses like secondary structure prediction, domain searching, phylogeny, etc. MSAProbs is currently the most accurate alignment algorithm, but its effectiveness is obtained at the expense of computational time. In the paper we present QuickProbs, the variant of MSAProbs customised for graphics processors. We selected the two most time consuming stages of MSAProbs to be redesigned for GPU execution: the posterior matrices calculation and the consistency transformation. Experiments on three popular benchmarks  on quad-core PC equipped with high-end graphics card show QuickProbs to be 5.7 to 9.7 times faster than original CPU-parallel MSAProbs. Additional tests performed on several protein families from Pfam database give overall speed-up of 6.7. Compared to other algorithms like MAFFT, MUSCLE, or ClustalW, QuickProbs proved to be much more accurate at similar speed. Additionally we introduce a tuned variant of QuickProbs which is significantly more accurate on sets of distantly related sequences than MSAProbs without exceeding its computation time. The GPU part of QuickProbs was implemented in OpenCL, thus the package is suitable for graphics processors produced by all major vendors. Multiple sequence alignment (MSA) is an essential task in molecular biology. It is performed for both, nucleotide and protein sequences. Its field of applications covers phylogenetic analyses, gene finding, identification of functional domains, prediction of secondary structures, and many others. Rapidly increasing size of sequence databases allowed by the development of high throughput sequencing technologies provides biologists with the opportunity to analyse in silico enormous sets of data. Hence, the constant pressure for developing more accurate and faster MSA algorithms. As multiple sequence alignment problem is NP-hard Estimate evolutionary distances between all pairs of sequences.Build a guide tree on the basis of the distances.Align sequences in the order described by the tree.Calculation of an evolutionary distance is done either by performing pairwise sequence alignment between sequences (default mode) or by employing Many researches aimed at refining ClustalW accuracy by extending the idea of progressive alignment. An important breakthrough was the introduction of T-Coffee algorithm Our research, however, focuses on methods exploiting only sequences themselves. An important issue related to accurate MSA algorithms like T-Coffee, ProbCons, or MSAProbs is that superior results are produced at the cost of significant increase in complexity: all the above-mentioned methods are inferior to ClustalW in fast mode in terms of time and memory requirements. This is a serious disadvantage when processing large sequence sets, which is often the case, as it has been proven that introducing homologous sequences to MSA improves quality of a final result For aforementioned reason, some algorithms aimed at improving alignment quality without sacrificing time and memory efficiency of ClustalW. These are for example Kalign Nevertheless, experiments clearly show that if alignment quality is of paramount importance, consistency-based methods are out of competition. In order to overcome their greatest disadvantage, i.e., large execution times, many algorithms utilise multi-core architecture of modern CPUs. One of the best examples is MSAProbs which assessed on quad-core CPU turned out to be faster than its less accurate serial competitors like ProbCons or Probalign. Yet, parallelisation on central processors has its limitations. Nowadays, a typical desktop PC is equipped with four- or six-core CPU and to further decrease execution times, expensive multi-processor architectures have to be used. One of the ways of addressing this issue is using a potential of graphics processors in general purpose computing. Since computational power of current GPUs is more than order of magnitude greater than power of central processors, developing GPU-suited versions of algorithms has become popular in many computational demanding tasks also in bioinformatics. One must keep in mind, that differences in architectures of graphics processors and CPUs are fundamental. GPUs have thousands of cores, several types of memory and utilise massively parallel execution model. This makes designing algorithms customised for GPUs a challenging task that cannot be accomplished by adapting serial or parallel methods destined for CPU execution.Heretofore, GPU customisation of multiple sequence alignment algorithms concerned mainly ClustalW The algorithm executes on GPU the most time consuming parts of MSAProbs pipeline, i.e., the posterior probability matrices calculation and the consistency transformation. These stages are parallelised in MSAProbs on CPU with a use of OpenMP forward-backward algorithm for pair-HMMs or partition function calculation. The dynamic programming (DP) was a subject of GPU customisation multiple times, also in bioinformatics. It concerned pairwise sequence alignment The posterior calculation stage executes dynamic programming methods like The consistency transformation stage incorporates a set of sparse matrix multiplications. There are algorithms and libraries for this task QuickProbs uses new, graphics processor specific, intra-task parallel algorithms for both, posterior matrix calculation and consistency transformation. Additionally, we parallelised on CPU the alignment construction and refinement stage, which in MSAProbs is performed serially. As a result, our package is several times faster than MSAProbs. This allows user to process larger datasets in a reasonable time without sacrificing alignment quality. Additionally we present a tuned variant of method called QuickProbs-acc. It significantly outperforms MSAProbs in terms of accuracy on sets of distantly related sequences without exceeding its running times.Let Computational power of current graphics processors is several times greater than power of CPUs. Rapid development of programming interfaces like CUDA processing elements (cores) gathered in several compute units (multiprocessors). Processing elements within compute units operate according to a single instruction-multiple data or single program-multiple data paradigm. From logical point of view a GPU program (known as a kernel in both OpenCL and CUDA) consists of many work-items (threads) gathered in workgroups (blocks). An important fact is that synchronisation between work-items can be done only within a workgroup. Thus, matching the number and the size of workgroups for a particular task is a crucial issue when developing GPU-suited algorithms. After execution of a kernel, a hardware scheduler maps work-items in the way that a workgroup is executed on a single compute unit, while one unit can handle multiple workgroups. OpenCL does not specify how workgroups are run by hardware but in order to efficiently utilise computational power of GPU, knowledge of workgroup execution at the device level is necessary. The smallest amount of work that is physically performed on AMD GPUs consists of 64 work-items and is called a wavefront; on NVidia devices it has 32 items and is known as a warp. Therefore, it is important to make the group size multiplicity of these portions. There is no guarantee in which order wavefronts (warps) are executed—this is decided by the scheduler dynamically. An important consideration is that all work-items within that portion must share exactly the same execution path. A divergence in a wavefront (warp) caused, e.g., by the presence of conditional statements is realised by executing instructions from all paths with some work-items being masked when necessary. Hence, data dependant branching inside wavefronts (warps) increases kernel execution time and should be avoided. Another important issue when developing algorithms on GPU is providing sufficient occupancy of a device. In order to hide delays of arithmetic instructions and, most importantly, memory accesses it is recommended to invoke a few times more work-items than the number of processing elements. On modern devices this results in as much as 104 work-items per kernel.The reason why designing algorithms suited for GPU execution is a challenging task is a great difference between architectures of central and graphics processors. Unlike CPUs that contain few cores, modern GPUs are composed of thousands of global memory which is an equivalent of main memory at CPU. Maximal throughput of global memory is several times greater than main memory bandwidth. However, it can be achieved only in the case of coalesced accesses when consecutive work-items utilise data from contiguous 128-byte area. In such situation whole portion of data can be read in one transaction. Since latency of global memory is large, non-coalesced accesses often result in lower bandwidth than at CPU. This issue has been partially solved in recent graphics processors by caching. Nevertheless, compared to CPUs which have several megabytes of cache per core, amount of cache on GPU is much smaller—hundreds of bytes per processing element. Another limitation concerning global addressing space is lack of virtual memory on GPU: programmer is responsible for fitting all necessary data in a limited storage area. Each compute unit contains additional amount (tens of kilobytes) of fast, directly addressable local memory (shared memory) which can be used for buffering frequently accessed data. Graphics processors contain much more registers than CPUs (tens of thousands per compute unit). The size of local memory and the number of registers allocated by work-items are important for execution time as they affect the maximum size of workgroup as well as GPU occupancy.An important difference between CPU and GPU concerns memory architecture. GPU is equipped with few gigabytes of Concluding, graphics processor execution model differs significantly from CPU. Presence of thousands of cores at GPU device requires fine-grained algorithm parallelisation. Programmer needs to take care of coalesced accesses to global memory to maximise throughput. Careful use of local memory and registers is necessary to keep GPU occupancy at desired level. Finally, calculations should be organised in the way, that eliminates branching within the same wavefront (warp). As a result, algorithms destined for graphics processors often use different data structures and computation schemes than methods suited for central processors, even those exploiting multi-core architectures. Due to this fact algorithms for GPUs must be designed and implemented from the scratch rather than be adopted from CPU.MSAProbs is currently the most accurate multiple sequence alignment software. It is a progressive strategy based on the following stages:I. Calculation of posterior probability matrices and distances for all pairs of sequences.II. Construction of a guide tree upon distances and calculation of sequence weights.III. Performing weighted consistency transformation on all posterior matrices.IV. Building final alignment using the guide tree and posterior matrices followed by the iterative refinement.The algorithm scheme can be found on exploiting maximum accuracy criterion during pairwise alignments,relaxing posterior matrices over other sequences,performing the iterative refinement.First two methods decrease probability of misaligning given pair of residues by utilising information from suboptimal pairwise alignments and alignments of other sequences. If misalignment occurs, third feature gives the opportunity to correct it.Predominance of MSAProbs over previously published methods is obtained at the cost of computation time. The most time consuming stages are those performed for all pairs of sequences, that is the posterior matrix calculation (I) and the consistency transformation (III). The worst-case time complexities of these stages are Posterior matrix calculation. This stage of MSAProbs consists of calculating set of posterior probability matrices for all pairs of sequences inter-task.After posterior matrix computation, the Needleman-Wunsch algorithm Tree construction and sequence weighting. The guide tree construction in MSAProbs is performed with a use of UPGMA algorithm. After that sequences are weighted in the order described by the tree using ClustalW weighting scheme. The weight of sequence Consistency transformation. The consistency transformation stage in MSAProbs relies in relaxing all posterior matrices calculated in stage I over sequences from An important note is that MSAProbs does not allow any new elements to be introduced to Final alignment construction and refinement. The last stage of all progressive methods is the computation of final alignment according to the guide tree constructed during stage II. At each tree node a weighted profile-profile alignment (referred to as a progressive step) is performed. When aligning two profiles Calculation of the final alignment is followed by the refinement, one of the classic techniques of progressive methods. In MSAProbs it is called a post-processing stage. It was designed as a solution for one of the greatest problems of progressive strategies—wrong choice of sequences or profiles to be aligned in early steps of the final result construction. Such misalignments cannot be corrected in the following progressive steps and affect overall accuracy of the method. The refinement tries to solve this problem by splitting final alignment horizontally into two random profiles. Then, the columns containing only gaps are removed and the profiles are realigned in the same way as in the final alignment stage. The aforementioned procedure is repeated In MSAProbs the final alignment and refinement procedures are executed serially.The main goal of our study was to speed up the most time-consuming stages of MSAProbs  by redesigning them for GPU execution using OpenCL. Additionally, we decided to parallelise on CPU the last stage of MSAProbs algorithm which previously was implemented in a serial manner. This is because the construction and refinement of final alignment, if executed serially, would be the most time consuming part of QuickProbs. In the following subsections we present how stages I, III, and IV were modified in QuickProbs. The separate subsection is devoted to QuickProbs-acc, a specialised variant of our package which aims at improving MSAProbs quality.Posterior matrix calculation. The posterior matrix calculation in QuickProbs is, similarly to MSAProbs, based on tasks. However, massively parallel architecture of graphics processors requires smaller portions of work to be done by a work-item. Therefore, it was assumed that each task is processed by a single workgroup. That computation scheme can be denoted as intra-task parallelisation (one task—many work-items). As work-items within workgroup can synchronise and exchange data through local memory, they are able to properly handle data dependencies present in a task. Another important difference is that modifications made in QuickProbs algorithm at the consistency stage result in the necessity of calculating calculate dense posterior matrix build sparse matrix build sparse transposed matrix Let At the beginning of posterior calculation stage, descriptions of all posterior tasks are prepared. As each task consists in computing two sparse matrices forward-backward algorithm Step (1) of the posterior task consists in computation of matrix and for forward and reverse passes, respectively, with different Note, that all intermediate matrices needed for osed see  in whichelement is used as a basic memory unit in the paper. Sometimes it is referred to as a dense element to distinguish it from sparse elements defined later. Pair-HMM used in MSAProbs contains 5 states. MSAProbs stores whole DP matrices for all states resulting in For convenience an sparse elements denoted as Steps (2) and (3) of the posterior task transform dense If column numbers and indices are of the same size as matrix elements, storing Assuming that elements in device memory. The space for storing posterior layers and output sparse matrices, i.e., Consistency transformation. QuickProbs version of consistency transformation differs from its predecessor. First of all, as GPU computations are strongly memory limited, QuickProbs operates directly on sparse representations. As a result, the elements of At the beginning of the consistency transformation QuickProbs generates descriptions of all stripes. There are subgroup indicate a set of work-items which processes a stripe. As single sparse element is analysed by a single work-item, there are The relaxation procedure performed at GPU starts from weighting input matrix by A detailed description of actions performed by a single subgroup is presented below. For convenience, let us introduce an additional notion: After considering all these factors, we decided to store The relaxation of elements. In our preliminary experiments performed on both NVidia and AMD GPUs the best results were obtained for The most important disadvantage of the described method is its sensitivity for divergence of sparse matrices widths. This effect is especially visible for large datasets and is caused by multiplication algorithm containing several data-dependant nested loops. Branch divergence within a wavefront (warp) in outer-loop caused by different lengths of rows is very expensive, as alternative execution paths are time consuming. This effect is partially reduced by using stripes, but not entirely removed. To avoid waste of computational power, the diversity of sparse row lengths have to be minimised. The simplest way of obtaining it is gathering in sectors Final alignment construction and refinement. In MSAProbs the last stage of the algorithm is executed serially, which is appropriate for small datasets when profiles contain few sequences. When the number of sequences is larger, the computation time of stage IV becomes comparable to the execution times of stages I and III. Since QuickProbs uses significantly faster algorithms for stages I and III, the last stage, if performed serially, would be the most time consuming part of the entire algorithm. Thus, we decided to parallelise this stage for multi-core central processors. There are two main parallelisation approaches possible.The first one follows inter-task concurrency scheme utilised by original MSAProbs for stages I and III. The Beside parallelisation, QuickProbs introduces a simple optimisation to the final alignment calculation. When MSAProbs computes In QuickProbs the refinement stage is also done in parallel with the use of the profile alignment inter-task algorithm described above.Accurate mode. QuickProbs, similarly to MSAProbs, has two parameters influencing quality of a final alignment, the number of consistency transformations The main disadvantage of approach based on global Experiments were performed on several hardware configurations built upon three CPUs  and four graphics cards . All computers used for the tests were controlled by Windows operating system. Detailed parameters of hardware used in the experiments can be found in The main tests were performed on three popular benchmarks containing amino acid sequences, i.e., BAliBASE 3 For each dataset we calculated sum-of-pairs (SP) and column-scores (TC) measures with a use of Qscore software The aim of the second experimental step was to compare base QuickProbs version and its accurate variant QuickProbs-acc with competing algorithms. SP and TC were used as quality measures and are reported together with total execution times in The first observation when analysing results for benchmark datasets from The shortest absolute execution times of analyses as well as highest speed-ups with respect to the original MSAProbs were obtained by Xeon W3550 + Radeon 7970 platform see , 9, 10. When detailed results for particular stages are analysed, the highest speed-ups are obtained for stage I. For Xeon W3550 + Radeon 7970 they vary from 13.3 (BAliBASE) to 19.2 (PREFAB). GeForce 680 turned out to be slower with speed-ups varying from 9.9 to 12.7. On hexa-core Xeon machine speed-ups are lower, but execution times are still superior to the original MSAProbs. Interesting observation comes from analysis of times obtained by GeForce 480 and 560, which are noticeably smaller than is suggested by the difference in computational power. After deeper investigation it became clear that the problem is caused by a number of registers in these GPUs which limits maximum size of a workgroup Experiments on real protein families were performed on our fastest testing platform equipped with Xeon W3550 CPU and Radeon 7970 GPU. Results are presented in Aligners from In cases where MSAProbs accuracy is sufficient, default mode of QuickProbs should be used as it produces almost identical results as MSAProbs, at a fraction of the time. Small discrepancy in accuracies is caused by using double floating-point precision in partition function calculation at GPU instead of long double. This was confirmed by the fact, that MSAProbs compiled without long double support gives exactly the same results as QuickProbs. Fourth aligner in terms of quality rank was MAFFT in automatic mode. Additionally, it turned out to be slower than QuickProbs on BAliBASE and PREFAB benchmarks.Second group of packages consists of algorithms without consistency: ClustalIn the paper we present QuickProbs, a variant of MSAProbs algorithm suited for graphics processors. We designed and implemented GPU versions of two most time consuming stages of the strategy, which originally were customised for multi-core architecture with the use of OpenMP. These are the calculation of posterior probability matrices and the consistency transformation. Posterior matrices are calculated on the basis of pair hidden Markov models and partition functions. From algorithmic point of view the stage performs several dynamic programming passes. Customising computation scheme to massively parallel GPU architecture, optimising global memory accesses by using jagged pattern and exploiting advanced method of work balancing resulted in significant speed-ups at this stage. On the main testing platform equipped with a quad-core Xeon W3550 and Radeon 7970, QuickProbs calculated posterior matrices as much as 24.7 times faster than original CPU-parallel method. The consistency transformation stage relies on performing set of small sparse matrix multiplications. We designed an algorithm for this purpose customised for graphics processors. As experiments on the basic testing platform show, it outperforms its MSAProbs equivalent with speed-ups reaching 8.0. In order to further improve execution times, we additionally suited the last stage of QuickProbs for multi-core CPU architectures with a use of OpenMP. Thanks to this, the construction and refinement of final alignment is done even 6.1 times faster than previously.Assessed on BAliBASE, PREFAB, and OXBench-X benchmarks, QuickProbs turned out to be respectively, 6.3, 9.7, and 5.7 times faster than MSAProbs. In the experiments on protein families from Pfam database, the overall speed-up was 6.7. This makes QuickProbs competitive to faster aligners like MAFFT, ClustalOur future plans focuses three main tasks: (1) removing limitation of sequence lengths that can be processed at GPU at posterior stage, (2) redesigning consistency transformation to make it less vulnerable for divergence in sparse rows lengths, (3) customising final alignment construction and refinement procedures for graphics processors.http://adaa.polsl.pl/agudys/quickprobs/quickprobs.htm. The detailed analysis of the time complexity of MSAProbs and QuickProbs can be found in the QuickProbs algorithm together with all the datasets used in the research are available at Supplement S1Detailed analysis of MSAProbs and QuickProbs time complexities.(PDF)Click here for additional data file.


Crumble breaks long alignment problems into shorter sub-problems. Prune divides the phylogenetic tree into a collection of smaller trees to reduce the number of sequences in each alignment problem. These methods are orthogonal: they can be applied together to provide better scaling in terms of sequence length and in sequence depth. Both methods partition the problem such that many of the sub-problems can be solved independently. The results are then combined to form a solution to the full alignment problem.Continuing research into the global multiple sequence alignment problem has resulted in more sophisticated and principled alignment methods. Unfortunately these new algorithms often require large amounts of time and memory to run, making it nearly impossible to run these algorithms on large datasets. As a solution, we present two general methods, Crumble and Prune, for breaking a phylogenetic alignment problem into smaller, more tractable sub-problems. We call Crumble and Prune Crumble and Prune each provide a significant performance improvement with little loss of accuracy. In some cases, a gain in accuracy was observed. Crumble and Prune were tested on real and simulated data. Furthermore, we have implemented a system called Job-tree that allows hierarchical sub-problems to be solved in parallel on a compute cluster, significantly shortening the run-time.These methods enabled us to solve gigabase alignment problems. These methods could enable a new generation of biologically realistic alignment algorithms to be applied to real world, large scale alignment problems. Multiple sequence alignment methods are a major tool in comparative genomics. The alignments they generate are primary data for a wide array of analyses: discovery of evolutionarily conserved elements ,2, identNew high throughput sequencing methods are greatly increasing the amount of available sequence information. To take full advantage of this wealth of data, current multiple sequence alignment methods will need to be adapted to handle larger datasets.We present two general methods to adapt current global alignment algorithms to large scale problems. This will enable current methods to be used for larger and larger problems and also allow computationally expensive methods to be applied to biologically relevant problems.Segmentation methods have been used to align full genomes. To align the mouse genome to the human genome, Schwartz et al. divided the human genome into approximately 3,000 segments of ~ 1.01 Mb with a 10 kb overlap between adjacent segments . They hyDress et al. describe a divide and conquer method that recursively subdivides three sequence alignment problems length-wise to produce shorter alignment problems . They ouk-mers shared by all sequences [k-mers are then aligned independently and concatenated to form the complete alignment. Crumble uses a more general method that does not require the constraints to span all sequences. Thus Crumble can be used to align more diverged sequences where it is impossible to find k-mers shared by all sequences. Crumble also takes advantage of parallelization while MISHIMA currently does not.The MISHIMA algorithm of Kryukov and Saitou starts by finding a non-conflicting set of equences . The seqThe method employed by Prune can be considered a generalization of the progressive alignment methodology ,10. ProgSeveral authors have extended specific alignment algorithms to take advantage of parallelization, usually thread or intra-core parallelization -13. ThesWe adopt the general approach of dividing a large alignment problem into smaller sub-problems. The sub-problems are solved and the alignments are recombined to form a solution to the whole, original problem. We perform this division in such a way that most sub-problems are independent and thus can be solved in parallel. This can lead to reduced run-time because of smaller individual problems and because sub-problems can be solved in parallel on a multi-core machine or cluster. Smaller problems can also result in less memory consumption; allowing some methods to be applied at all.Crumble deals with problems that are large in sequence length. It breaks up long alignment problems into shorter problems. Prune handles problems with a large number of sequences. It cuts up deep alignment problems into sub-problems with fewer sequences.An alignment problem can be large in two ways: large in sequence length or large in sequence number. meta-alignment methods. In theory, both these methods would result in performance gains at the loss of some alignment accuracy. However, we found that very significant performance gains can be made with negligible loss of alignment accuracy and in a few cases some accuracy can be gained.Our methods do not perform the actual alignment of sub-problems. Other programs are used for that part of the process. Almost any current global alignment algorithm can be adapted to work with Crumble and Prune by writing a simple wrapper. Thus we refer to Crumble and Prune as x in a sequence, Crumble searches for a set of base positions A = {a1, a2, ..., na} in each of the n species being aligned such that x ≼ i afor all i  in it. The only sub-trees rooted at a leaf that make sense are those containing one node. Thus Equation (1) assigns all sub-trees rooted at leaf i containing only one sequence a stage count of 1. This trivial "alignment" of a single sequence is counted as a stage since time is spent extracting and processing the sequence. Equation (2) assigns all other trees below leaf i (but not rooted at i) a stage count of ∞. Equation (3) calculates the minimum number of stages needed if node i is the root of a sub-tree. It does this by considering all ways to merge the right (ir) and left (il) children of node i into a new sub-tree while respecting the maximum number of sequences per sub-tree, M. It selects the merge that minimizes the number of stages at i. The new stage depth becomes one more than this minimum. Equation (4) calculates the minimum number of stages needed if the sub-tree below node i (but not rooted at i) contains exactly j sequences, leaf or inferred. Thus, SR1 is the minimum number of stages needed for the entire tree if the global root node R is the root of a sub-tree. The above recurrence relation can be calculated using a dynamic programming algorithm and the sub-trees that achieve the minimum number of stages can be inferred. For a given maximum number of sequences per sub-tree, the dynamic programming calculation takes time linear in the number of species.where Maximal, a heuristic that assumed that the most commonly occurring base in an alignment column was the base for that position in the root. Maximal is such named because it infers the longest possible root sequence that fits within the alignment  and thus not useful for testing Crumble. Thus, we used a different method to evaluate Crumble on long alignment problems. We extracted and aligned a 90 kb region of the human genome and the orthologous region from six other species. To get an idea of alignment accuracy, we calculated the log-likelihood of the alignment. We used phyloFit from the PHAST package to calculate the maximum log-likelihood of the alignment . While tThe computations in this section were performed on a different cluster than the one used for the results of Section. That cluster has been reappropriated. The cluster used for this section has eight nodes each with two dual-core AMD Opteron 2214 HE running at 2.2 GHz and 32GB of memory.bosTau4 assembly), dog (canFam2), mouse (mm9), chimp (panTro2), macaque (rheMac2), and rat (rn4) and extracted the corresponding sequence to form the alignment problem.We examined Crumble's effect on performance by aligning genomic DNA from 7 species. Looking at the UCSC Genome browser, we selected a ~90 kb region on the human genome, chr14:104721193-104812803 . This reAs above, Crumble was used to break the alignment problem into several fractional sized pieces Table  and PecatRNA, 5S_rRNA, SRP_bact, MIR807, Cobalamin, PK-G12rRNA, SSU_rRNA_5, RNaseP_bact_a, tmRNA, SAM, U2, and U6. We tested the generated alignments against the hand curated "seed" alignment maintained by Rfam using the agreement score  and deep (1000 species) alignment problems. Crumble and Prune were able to dramatically improve the run-time of Pecan, FSA, and MUSCLE on long and deep alignment problems with very little loss in alignment accuracy. In some cases, Prune was also able to improve the accuracy of FSA and MUSCLE while, at the same time, providing a boost in performance. We also tested Crumble and Prune on a set of biological data. While these data sets are relatively small with respect to the scale of problems that Crumble and Prune are designed to handle, they show that our methods do provide a significant performance improvement with only moderate loss of accuracy.With Crumble and Prune we were able to apply Pecan, FSA, and MUSCLE to much longer and deeper problems than could be solved by running those programs without Prune because of memory or time constraints. This extends the applicability of these methods to larger alignment problems.We believe that these methods will enable the application of more sophisticated and statistically motivated alignment algorithms toward large, real world alignment problems.• Project name: Crumble, Prune, Job-treehttp://hgwdev.cse.ucsc.edu/~krish/crumble_prune/. Datasets used to test the alignments are available at: http://hgwdev.cse.ucsc.edu/~krish/test_alignments/.• Project home page: • Operating System(s): Linux 2.6.18• Programming language: C++, Python• Other requirements: Boost C++ Libraries 1.46• License: GNU GPLhttp://hgwdev.cse.ucsc.edu/~krish/crumble_prune/. The datasets used to test the alignments are available at: http://hgwdev.cse.ucsc.edu/~krish/test_alignments/.Crumble, Prune, and Job-tree are licensed under the GPL and and available for download at: KMR wrote the implementation, participated in software design, performed the benchmarking, and drafted the manuscript. BP participated in software design, assisted with the implementation, and helped draft the manuscript. DH participated in the design and coordination. All authors read and approved the final manuscript.


Pairwise statistical significance has been recognized to be able to accurately identify related sequences, which is a very important cornerstone procedure in numerous bioinformatics applications. However, it is both computationally and data intensive, which poses a big challenge in terms of performance and scalability.© Core™i7 CPU 920 processor.We present a GPU implementation to accelerate pairwise statistical significance estimation of local sequence alignment using standard substitution matrices. By carefully studying the algorithm's data access characteristics, we developed a tile-based scheme that can produce a contiguous data access in the GPU global memory and sustain a large number of threads to achieve a high GPU occupancy. We further extend the parallelization technique to estimate pairwise statistical significance using position-specific substitution matrices, which has earlier demonstrated significantly better sequence comparison accuracy than using standard substitution matrices. The implementation is also extended to take advantage of dual-GPUs. We observe end-to-end speedups of nearly 250 (370) × using single-GPU Tesla C2050 GPU  over the CPU implementation using IntelHarvesting the high performance of modern GPUs is a promising approach to accelerate pairwise statistical significance estimation for local sequence alignment. The past decades have witnessed dramatically increasing trends in the quantity and variety of publicly available genomic and proteomic sequence data. Dealing with the massive data and making sense of them are big challenges in bioinformatics ,2. One os1 and s2 of lengths m and n, respectively, the scoring scheme, SC , and the number of permutations N of s2, pairwise statistical significance (PSS) of the two sequences is calculated by the following function [A], S[A][K], S[A][L], S[A][G] are usually stored far from each other. In contrast, if we use CUDA built-in vector data type int4 (which packs 4 integers together) to store the customized substitution matrix in the texture memory, the same four scores are stored at neighboring memory locations  against query sequence residues  are required subsequently, GPU has to look up the matrix four times one by one, because the score In essence, both coalesced memory layout and customized substitution matrix optimization heighten performance of memory access by improving data locality among threads in GPU programming.s2 copy has to be permuted thousands of times. Although the most widely used pseudorandom number generators such as linear congruential generators (LCGs) can meet our requirements, the current version of CUDA does not support calls to the host random function. Hence, we develop an efficient random number generator similar to lrand48 on GPUs.As mentioned previously, to simulate the required random sequences, a lot of random numbers are needed, since each s1 and the subject sequence s2, to compute PSSE, thousands (say N) of randomly permutations of s2 are needed. To obtain these N random sequences of s2, first, a set of N random numbers is generated in CPU as described in previous section. These numbers are then transferred to GPU and considered as seeds by the threads of GPU. Each thread then generates new random numbers using its own seed and swap the symbols of s2 accordingly to obtain a permuted sequence. Thus the N random permutations of s2 are obtained in parallel. The algorithm then uses SW algorithm to compute alignment score of s1 and the N permuted copies of s2 in parallel on GPU. The scores are then transferred to CPU for fitting. The details of computing single-pair PSSE is outlined by Algorithm 1. Note that Step 4 in Algorithm 1 uses the optimized memory layout, explained previously in detail, for s2 and its N permuted copies.The single-pair PSSE processes only one pair of query and subject sequences. The idea of computing single-pair PSSE is as follows. Given the query sequence inter-task parallelism. In this case, each thread performs alignment of one pair of sequences. Hence, in a thread block, multiple alignment tasks are performed in parallel [intra-task parallelism. Here, alignment of each pair of sequences is assigned to a block of threads, splitting the whole task into a number of sub-tasks. Each thread in the thread block then performs its own sub-tasks, cooperating to exploit the parallel characteristics of cells in the anti-diagonals of the local alignment matrix [Smith-Waterman is a dynamic programming algorithm to identify the optimal local alignment between a pair of sequences. In general, there are two different methods for parallelizing the alignment task . The firparallel . The sect matrix . We use t matrix  with somThe multi-pair PSSE processes multiple queries and subject sequences. We can obtain some hints about improving the performance by analyzing the single-pair PSSE implementation (which we do in a later section). Owing to a dramatic increase of data set for multiple-pairs implementation, there are some significant differences relative to performance of GPU hardware between the two implementations. Through analyzing our experimental results of the single-pair PSSE, we observe that inter-task parallelism performs better than intra-task parallelism (results shown later). Hence, we consider inter-task parallelism in computing multi-pair PSSE. Based on the guidelines for optimizing memory and occupancy described earlier, we compare three implementation strategies.Algorithm 1: Pseudo-code of single-pair PSSEInput:  - Sequence-pair; M - Substitution matrix; G - Gap opening penalty; GE - Gap extension penalty; N - Number of permutes;Output: pss- Pairwise statistical significance1. InitializationN of random numbers in CPU;(a) Generate a number (b) Copy LCG seeds to GPU global memory;s1 and s2 to GPU global memory;(c) Copy 2. Generate random numbers (RNs) using LCG in GPUs2 N times using the RNs3. Permute sequence s2 and its N times permuted copies as shown in Figure 4. Reorganize sequences s2 and its N permuted copies against s1 using Smith-Waterman (inter-task or intra-task);5. Align N alignment scores from GPU into CPU;6. Transfer 7. Fitting in CPUK, λ) ← EV DCensordMLFit(Scores);(a) (pss ← 1 - exp(-Kmne -λx);(b) return pssQ query sequences and S subject sequences, the intuitive strategy is to simply perform the same 'single-pair' procedure Q × S times. In other words, in each iteration, we send a single pair of query and subject sequences to the GPU. The GPU processes that pair and returns the result to the CPU. The same procedure is repeated for all query and subject sequence pairs. However, this strategy suffers from low occupancy. We analyze the cause along with its performance results in the next section.Given In the first strategy, the subject sequences from the database are permuted for every query-subject sequence pair. Hence, the same subject sequence is permuted every time it is sent to the GPU. A better strategy is to create permutations of each subject sequence only once and reuse them to align with all the queries. "One permutation, all queries" is the idea of the second strategy. Because of the reuse of permuted sequences, higher performance is expected than the first strategy. However, the occupancy of GPU, to be shown in the next section, is still not elevated.The low occupancy of the above two strategies is due to the underutilized computing power of GPU. In addition, these two strategies do not work well when the size of subject sequence database becomes too big to be fitted into GPU global memory. For instance, if the size of the original subject sequence database is 5 MB, it becomes 5000 MB when each of the sequences is permuted, say, 1000 times. This prohibits transfer of all the subject sequences to GPU at the same time. We therefore need an optimal number of subject sequences to be shipped to GPU keeping in mind that the subject sequences and their permuted copies fit in global memory. Moreover, the number of new generated sequences should be enough to keep all CUDA cores busy, i.e., keep a high occupancy of GPU, which is very important to harness the GPU power.tiles and transferred to the GPU global memory. In our case, the tiles are the number of subject sequences to be transferred to the GPU at a time. The tile size T can be calculated using the following equation:Herein we develop a memory tiling technique that is self-tuning based on the hardware configuration and can achieve a close-to-optimal performance. The idea behind the technique is as follows. In out-of-core fashion, the data in the main memory is divided into smaller chunks called num SMis the total number of SMs in GPU, max Tis the maximum number of resident threads per SM, and N is the number of permutations.where N = 1000, then, T = ⌊14 × 1024/1000⌋ = 14, which means that there are 14 distinct subject sequences to be transferred to the GPU's global memory at a time. Based on the second strategy , 14 subject sequences and their permuted copies are aligned against one query sequence at a time, until all the query sequences are processed. As a result, 14 × 1000 alignment scores in total are obtained in each round, which are subsequently transferred to CPU for fitting. The CPU takes the 1000 alignment scores for each subject-query sequence pair and uses them to compute the corresponding pss. The tile-based strategy has been described in Figure In Tesla C2050 used in our experiments, there are 14 SMs and the maximum number of resident threads per SM is 1024. Let © Core™i7 CPU 920 processors running @2.67 GHz. The system has 4 cores, 4 GB of memory, dual Tesla C2050 GPU (each with 448 CUDA cores) and is running a 64-bit Linux-based operating system. Our program has been compiled using gcc 4.4.1 and CUDA 4.0. The sequence data used in this work comprises of a non-redundant subset of the CATH 2.3 database [k for a gap of length k. We permute the 2771 subject sequences N = 1000 time as in several previous studies [All our experiments have been are carried out using Intel studies ,35,36,45In single-pair PSSE implementation, we use both the intra-task and inter-task parallelism methods. We choose four pairs of query and subject sequences of length 200, 400, 800, and 1600 from CATH 2.3 database. We compute PSSE for all these pairs using 64, 128, 256, and 512 threads per block. The experimental results have been plotted in Figure For intra-task parallel implementation, the best speedups for sequences of length 200, 400, and 800 are 24.57×, 35.09×, and 43.63×, respectively. All these are obtained using 128 threads per block. But the best speedup for sequences of length 1600 is 46.34× and used 256 threads per block. These results tell us that it is hard to find a general rule to parameterize the settings to achieve peak performance. A possible reason is that many factors, such as the number of threads per block, the available registers, and shared memory, may contradict with each other.The intra-task parallel implementation creates enough thread blocks  to keep the occupancy high. However, the SW alignment matrix must be serially computed from the first to the last anti-diagonal. Only the cells of the alignment matrix belonging to the same anti-diagonal can be computed in parallel. In this case, most threads in a block have no work to do. As a result, the performance of the intra-task parallel implementation is worse than that of the inter-task parallel implementation even though it has higher occupancy.B = ⌈1000/64⌉ = 16. The assignment of 16 blocks to 14 available SMs will result in 12 SMs with one active block and two SMs with two blocks. Therefore, the occupancy for the two SMs with two blocks is (2 × 64)/1024 = 12.5%. For the 12 SMs with only one block its occupancy is (1 × 64)/1024 = 6.25%. Because of the low occupancy, there are not sufficient threads to keep CUDA cores busy when the global memory is accessed. As a result, the latency-hiding capabilities of this method are limited. As the number of threads per block num Tincreases, the total active blocks (B) decreases and some SMs even have no blocks assigned. For example, if num T= 512, then B = ⌈1000/512⌉ = 2. Hence only two SMs are working with one block each.In contrast, inter-task parallelism would have a total of 1000 threads . If each block contains 64 threads, then the total number of blocks is However, these SMs have a higher occupancy (1 × 512)/1024 = 50%, which compensates for the decrease in the number of working SMs. Consequently, even though there is a reduction in speedups, it is not directly proportional to the reduction in the number of active SMs. When the number of threads per block is 64, for the sequence length of 200, 400, 800, and 1600, the best speedups are 52.14×, 56.36×, 73.39×, and 73.16×, respectively.In brief, getting performance out of a GPU is about keeping the CUDA cores busy. Both inter-task parallelism and intra-task parallelism, under the situations of small data set being processed, seem to fail in this regard, but not necessarily for the same reason.Q, the number of query sequences. To alleviate the low occupancy problem, our proposed tile-based strategy uses a carefully tuned tile size that effectively increase the occupancy. Recall that, as a result of tiling, we send 14 subject sequences and one query sequence to the GPU in each round. After the permutation step, we have 14 × 1000 alignments to be performed, assuming each subject sequence is permuted 1000 times. Consequently, each SM has 1000 alignments to perform. Also, note that, the maximum number of blocks that can be launched simultaneously on an SM is 8. Therefore, when the number of threads per block is 64, the number of blocks per SM is min = 8, resulting in an occupancy of (8 × 64)/1024 = 50%, based on Equation (2) and (3). This is a significant improvement over the intuitive and data-reuse strategies. The maximum theoretical occupancy of an SM for the three strategies is given in Table In the intuitive and data reuse strategies, most of the SMs suffer from the same low occupancy as the single-pair PSSE implementation using the inter-task parallelism method. As expected, we observe poor performance for both strategies, as shown in Figure Due to a high occupancy, the tile-based strategy using single  GPU(s) achieves speedups of 240.31 (338.63)×, 243.51 (361.63)×, 240.71 (363.99)×, and 243.84 (369.95)× using 64, 128, 256, and 512 threads per block, respectively, as shown in Figure The billion cell updates per second (GCUPS) value is another commonly used performance measure in bioinformatics . The tilIn summary, low occupancy is known to interfere with the ability to hide latency on memory-bound kernels, causing performance degradation. However, increasing occupancy does not necessarily increase performance. In general, once a 50% occupancy is achieved, further optimization to gain additional occupancy has little effect on performance . Our expK and λ, and the same algorithm parameters) as in [http://cucis.ece.northwestern.edu/projects/PSSE/.Since the GPU implementation presented in this paper uses the same algorithm for PSSE  as in , the rets) as in . Accordis) as in , pairwiss) as in . The impIn this paper, we present a high performance accelerator to estimate the pairwise statistical significance of local sequence alignment, which supports standard substitution matrix like BLOSUM62 as well as PSSMs.Our accelerator harvests the computation power of many-core GPUs by using CUDA, which results in high end-to-end speedups for PSSE. We also demonstrate a comparative performance analysis of single-pair and multi-pair implementations. The proposed optimizations and efficient framework are applicable to a wide variety of next-generation sequencing comparison based applications, such as, DNA sequence mapping and database search. As the size of biological sequence databases are increasing rapidly, even more powerful high performance computing accelerator platforms are expected to be more and more common and imperative for sequence analysis, for which our work can serve as a meaningful stepping stone.BLAST: Basic Local Alignment Search Tool; BLOSUM: BLOcks of Amino Acid SUbstitution Matrix; CUDA: Compute Unified Device Architecture; EVD: Extreme Value Distribution; GCUPS: Billion Cell Updates Per Second; GPU: Graphics Processing Unit; PSSM: Position Specific Scoring Matrix; PSSE: Pairwise Statistical Significance Estimation; PSI BLAST: Position Specific Iterative BLAST; PSA: Pairwise Sequence Alignment; PSS: Pairwise Statistical Significance; SIMT: Single: Instruction, Multiple: Thread; SSM: Standard Substitution Matrix SM: Streaming Multiprocessor; SW: Smith-Waterman.The authors declare that they have no competing interests.YZ conceptualized the study, carried out the design and implementation of the algorithm, analyzed the results and drafted the manuscript; SM, AA, MMAP, WL, ZQ, and AC participated analysis of the results and contributed to revise the manuscript. All authors read and approved the final manuscript.