While previous word-frequency approaches calculated rough measures of sequence similarity or dissimilarity, some new alignment-free methods are able to accurately estimate phylogenetic distances between genomic sequences. One of these approaches is Filtered Spaced Word Matches. Here, we extend this approach to estimate evolutionary distances between complete or incomplete proteomes; our implementation of this approach is called Prot-SpaM. We compare the performance of Prot-SpaM to other alignment-free methods on simulated sequences and on various groups of eukaryotic and prokaryotic taxa. Prot-SpaM can be used to calculate high-quality phylogenetic trees for dozens of whole-proteome sequences in a matter of seconds or minutes and often outperforms other alignment-free approaches. The source code of our software is available through Github:  Here, groups of orthologous sequences have to be identified first, for which then multiple alignments are to be calculated. There are generally two different strategies of resolving phylogenies based on multiple alignments. In the so-called supermatrix approach, multiple sequence alignments of single genes or proteins are concatenated. A phylogenetic tree is inferred from the resulting matrix, kelihood  or Bayeskelihood . Alternakelihood  or superkelihood  approachAll of these steps are time consuming, and manual intervention is often required. Therefore, word-based or alignment-free alternatives have been proposed recently, which are much faster and require much less data preparation. Most alignment-free methods compare the word composition of sequences , 9, 10, Accurate alignment-free tools are urgently needed because of the huge volume of data generated by new sequencing techniques. Another advantage of alignment-free methods, compared to alignment-based approaches, is the fact that they can be applied to incomplete data, e.g., to unassembled sequencing reads or to partially sequenced genomes . Note thad hoc measures of sequence similarity or dissimilarity. They are not based on stochastic models of molecular evolution, and they do not try to estimate distances between sequences in a statistically rigorous way. More recently, some alignment-free approaches have been proposed that are based on explicit models of DNA evolution. These methods are able to estimate the number of substitutions per site that have happened since two nucleic-acid sequences have evolved from their last common ancestor [The above-mentioned approaches to alignment-free sequence comparison calculate ancestor , 34, 35.A main application of alignment-free approaches is comparison of whole genomes. Consequently, most alignment-free methods have been designed to work on DNA sequences. If distantly related species are studied though, phylogenetic trees are usually inferred from protein sequences rather than from DNA sequences. The reason for this is that protein sequences are more conserved than DNA sequences, as synonymous substitutions are not visible in proteins. Thus, for distal species, it may be hard to detect similarities between genes at the DNA sequence level, while homologies may still be detectable among protein sequences. It is therefore highly desirable to have accurate alignment-free software tools that work on protein sequences, in addition to the available tools for DNA sequence comparison. Generic word-frequency methods can be applied to both DNA and protein sequences; the program Feature Frequency Profile (FFP), e.g., has been used for whole-proteome comparison . As mentHere, we propose an alignment-free method that estimates the phylogenetic distance between two taxa based on the average number of amino-acid substitutions in the whole proteoms since they evolved from their last common ancestor. Our approach is based on filtered spaced word matches (FSWM), a concept we introduced recently for whole-genome sequence comparison ; see 3131, 35 foTo evaluate our approach, we used simulated protein sequences and real-world whole proteomes. Test runs on the simulated sequences show that our distance estimates are very close to the true distances for distance values of up to around 2.0 substitutions per sequence position. On the real-world sequences, we evaluated our approach indirectly by phylogenetic analysis, as is common practice in the field. We used Prot-SpaM to estimate pairwise distances for various sets of taxa, and we applied the neighbor-joining algorithm  to calcuP of length ℓ – , i.e., for a length-ℓ word P over {0, 1} –, a spaced-word with respect to P is a length-ℓ word W over the alphabet W(i) = * if and only if P(i) = 0. An index i ∈ {1, …, ℓ} is called a match position of P or W, respectively, if P(i) = 1, and a don’t care position otherwise. The number of match positions in a pattern or spaced-word is called its weight w. We say that a spaced word W with respect to P occurs in a sequence S at some position i if one has W(k) = S(i + k − 1) for all k ∈ {1, …, ℓ} with P(k) = 1 – ,i.e., for all match positions of P.We consider sequences over an alphabet P between two sequences S1 and S2 at  if the same spaced word w.r.t P occurs at position i1 in S1 and at position i2 at S2. In other words, there is a spaced-word match between S1 and S2 at  if and only if one has S1(i1 + k − 1) = S2(i2 + k − 1) for all match positions k of P. Below is an example for a spaced-word match between two sequences S1 and S2 at  with respect to the pattern P = 1100101; the spaced word TN**D*P occurs at position 2 in S1 and at position 3 in S2:Moreover, we say that there is a spaced-word match w.r.t. Similar to our original FSWM approach, we estimate distances between protein sequences based on selected spaced-word matches between them, with respect to one or several pre-defined patterns. Distance values are obtained by comparing the amino-acid residues that are aligned to each other at the don’t-care positions of the selected spaced-word matches. This is similar to estimating distances in standard alignment-based approaches; the only difference from those standard approaches is that we are using don’t-care positions of spaced-word matches instead of full sequence alignments.T, then a spaced-word match is considered a random match and is not further considered. As a default, we use a threshold value of T = 0. To see that this threshold accurately separates homologous from background spaced-word matches, one can plot the number of spaced-word matches with a score s against s, see Fig. -match histogram, or spamogram for short. In these plots, two peaks are typically visible, a peak on the right-hand side representing homologous spaced-word matches and a peak on the left-hand side representing background matches. By default, we are using patterns with a weight of w = 6 and with 40 don’t-care positions, i.e., with a length of ℓ = 46.To estimate distances in this way, one has to make sure that only those spaced-word matches are selected that represent homologies, i.e., that the involved spaced-word occurrences go back to the same origin in the last common ancestor of the two proteins that are compared. To distinguish such "homologous" spaced-word matches from random background matches, we calculate a score for each spaced-word match using the BLOSUM62 substitution matrix . SimilarS1 and S2 are compared and a spaced word W occurs n time in S1 and n′ times in S2, then this gives rise to a total of n × n′ spaced-word matches. Taking all of these spaced-word matches into account for phylogeny reconstruction would overemphasize repeated regions where the same spaced words occur multiple times. Instead of using all possible spaced-word matches, we therefore use a one-to-one mapping of spaced-word occurrences in the compared sequences. That is, we ensure that each spaced word occurrence is involved in at least one of the selected spaced-word matches. Formally, if there are two spaced word matches at  and at , respectively, then we can include both of them simultaneously in our list of selected spaced-word matches only if W, we calculate the scores of all spaced-word matches involving W. We then select them one-by-one in descending order of their scores, always ensuring that each occurrence of W is used in at least one of the selected spaced-word matches.Moreover, we use a one-to-one mapping of spaced-word occurrences. Note that if sequences eviously . For a gFinally, in order to estimate pairwise distances between two input sequences, we consider the pairs of amino acids aligned to each other at the don’t-care positions of the selected spaced-word matches. Here, we are using the Kimura model  that appP. More precisely, we are using a set m binary patterns such that all patterns in w but have their match and don’t-care positions arranged differently. We then use spaced-word matches with respect to all patterns m = 5 patterns. To find suitable pattern sets, we integrated the tool rasbhari [The accuracy and statistical stability of the described approach depends on the number of selected spaced-word matches; the more matches we obtain, the more accurate and stable the results of our method will be. To increase the number of spaced-word matches, the default version of our program uses multiple patterns instead of one single pattern rasbhari  into ourrasbhari  of patteT = 0 to discard background spaced-word matches, and sets of m = 5 patterns. We compared our program to the following four other alignment-free methods that can be run on protein sequences: Average Common Substring Approach (ACS) [k = 0. The competing tools were used with their default parameters. In addition to evaluating these tools on protein sequences, we ran filtered spaced word matches on the complete genome sequences of the same taxa. All test runs were done on a 10 x Intel(R) Xeon(R) central processing using E7-4850 with 2.00 GHz with four cores each, equaling 40 cores and 1,000 GB of random access memory.To assess the quality of our new approach and to compare it to other alignment-free methods, we used artificially generated as well as real-world protein sequences. For the test runs, we used the default parameters of our program, namely, 6 match positions and 40 don’t care positions, i.e., a total pattern length of 46 –, a threshold of ch (ACS) , FFP [36ch (ACS) , kmacs [ch (ACS) , and CVTch (ACS) . Here, wm = 5 patterns , we did a second series of test runs with m = 1, i.e., with single patterns. Figure To evaluate the distances estimated by our program, we simulated sequences with the tool pyvolve . Pyvolveentclass1pt{minimaNext, we applied the above alignment-free methods to calculate phylogenetic trees from real-world protein sequences. For four different groups of species, we downloaded all available protein sequences from GenBank , in addin taxa is 2 · n − 6 [As mentioned above, Prot-SpaM uses a probabilistic algorithm to generate pattern sets, so the results of different program runs on a sequence set can differ slightly, even if the same parameter values are used. We therefore performed 100 program runs on each dataset. Table  · n − 6 . Program · n − 6 . Neighbo · n − 6 .Escherichia coli and Shigella. For each strain, we were able to download about 4,000-5,000 protein sequences; the total size of this dataset is around 41 MB. Figure Our first dataset consists of 29 strains of Wolbachia strains, a group of Alphaproteobacteria that are intracellular endosymbionts of arthropods and nematodes [Wolbachia, 16 distinct genetic lineages (“supergroups”) are currently distinguished (named by letters A-F and H-Q) that may differ in host specificity and type of symbiosis [Wolbachia I). A tree generated with Prot-SpaM from this dataset is shown Fig. As a second test case for benchmarking, we analyzed the phylogeny of ematodes . Within ymbiosis . We re-aymbiosis , therebyWolbachia benchmarking dataset, we analyzed relationships between supergroups based on available (draft) genomes see below (Wolbachia II). For within supergroup relationships (Wolbachia I), a program run of Prot-SpaM on the whole proteome recovered a tree that is largely congruent in topology and branch lengths in comparison to a phylogenomic supermatrix analysis of 252 single-copy orthologs that excluded genes that showed signs of recombination. A comparison based on RF distances showed that our new method outcompetes other available alignment-free programs  proteomes for all available Wolbachia draft and fully assembled genomes  of the binary patterns and spaced words, their length ℓ, the number m of different binary patterns used by the program, and the cutoff value T to separate homologous from background spaced-word matches. To see how these parameters influence the results of our software and to find suitable default values, we ran Prot-SpaM with varying values of these four parameters. Here, we modified one parameter at a time, using the respective default values of the remaining three parameters. The results are summarized in Tables w, ℓ, and T that work best for all datasets, but our default values seem to be a reasonable compromise. Using sets of m = 5 binary patterns does not improve the quality of the produced trees in terms of their RF distances to the reference trees compared to program runs with single patterns. Table E. coli/Shigella dataset where most taxa are closely related to each other and another one in the Metazoan dataset that contains taxa with very large evolutionary distances. In the pairwise comparison of E. coliO157:H7 strain EDL933 with E. coliO157:H7 Sakai (EHEC), Prot-SpaM selected more than 6,000,000 spaced-word matches. These two proteomes have less than 1,600,000 amino acids each, so in this case >3.75 spaced-word matches per sequence position were selected. By contrast, less than 13,000 spaced-word matches were selected in the comparison of Brugia malayi and Homo sapiens. The latter proteome has a length of more than 75,000,000 amino acids, so here less than 0.00017 spaced-word matches per sequence position were selected.The number of spaced-word matches in a pairwise sequence comparison depends on how similar the two sequences are to each other; see  for detak-mer frequencies or on the length of common substrings. These approaches have been applied not only to DNA but also to protein sequences. A drawback of these methods is that they can only calculate rough measures of sequence similarity or dissimilarity; they do not estimate phylogenetic distances in a rigorous way. More recently, word-based methods have been developed that can accurately estimate phylogenetic distances between genomic sequences based on stochastic models of DNA evolution. One of these approaches is FSWM.A number of so-called alignment-free approaches have been proposed in recent years to rapidly calculate phylogenetic distances between genomic sequences. Earlier approaches are based on In this study, we introduced Prot-SpaM, a new implementation of FSWM to compare complete or incomplete proteome sequences to each other. To our knowledge, Prot-SpaM is the first tool that can accurately estimate phylogenetic distances between protein sequences without the need to calculate full sequence alignments. Our benchmark results show that distance estimates obtained with our approach are accurate for a large range of phylogenetic distances. Distances calculated with CVTree,ACS,FFP, and kmacs, by comparison, are monotonously increasing with the number of substitutions between the compared sequences. The obtained distance values are far from proportional to the real distances, and they flatten out somewhere between 0.5 and 1.5 substitutions per position see Fig. . By contk-mer length of k = 4, so we used this value in our study. It has been reported, however, that FFP may perform better on protein sequences if larger values of k are used [In our program evaluation, we used all competing software tools with their respective default parameters, if such default values were recommended by their developers. It should be mentioned, however, that some of the evaluated programs might produce better results with different parameter settings. The program FFP, e.g., uses a default are used . A comprE.coli/Shigella and plant datasets, as shown in Table Wolbachia datasets, it still performed reasonably well and was again superior to competing approaches on the whole-proteome sequences, but it was outperformed by word-frequency methods on the 252 selected orthologous proteins. A possible explanation of this result is discussed below. On the large prokaryote dataset and for the metazoan set, by contrast, none of the compared programs could reproduce the reference trees that we used in our evaluation. These are difficult datasets since they span very large evolutionary distances. Also, it should be mentioned that there are no absolutely reliable reference trees available for these datasets. For the metazoan dataset, e.g., the position of the ctenophores is still a matter of debate [Prot-SpaM produced high-quality trees and was superior to other alignment-free methods for the f debate , 74, 75.An interesting result is the performance of Prot-SpaM compared to our previous approach FSWM that takes genomic sequences as input. For most groups of taxa in our study, the results of Prot-SpaM and FSWM were of similar quality in the sense that the RF distances to the reference trees were comparable for both approaches. However, for the set of 813 prokaryote taxa, our new spaced-words approach performed better on whole proteomes than our previous approach on whole genomes, as shown in Fig. Wolbachia I, we actually obtained better RF distances with Prot-SpaM and FSWM when we applied these programs to the whole-protein or whole-genome sequences, respectively, than when we applied them to the 252 selected orthologous proteins or to the genes coding for those proteins : linuxProgramming language: C++Other requirements: noneLicense: GNU GPLAny restrictions to use by non-academics: nonehttp://projects.gobics.de/data/protspam/paperData.tgz. Additional supporting data and snapshots of the code are available in the GigaScience repository, GigaDB [The sequence datasets and trees used for the program evaluation can be downloaded from , GigaDB ACS: Average Common Substring Approach; BLOSUM: Blocks Substitution Matrix; CVTree: Composition Vector Tree; FFP: Feature Frequency Profile; FSWM: filtered spaced word matches; indel: insertions and deletions ; kmacs: k-Mismatch Average Common Substring Approach; PHYLIP: PHYLogeny Inference Package; Prot-SpaM: proteome-based spaced-word matches; rasbhari: Rapid Approach for Seed optimization Based on a Hill-climbing Algorithm that is Repeated Iteratively; RF: Robinson-Foulds; spamogram: Spaced-Word-Match Histogram.The authors declare that they have no competing interestsThe project was partially funded by the Volkswagen Foundation, project VWZN3157. We acknowledge support by the Open Access Publication Funds of the Göttingen University.Wolbachia and wrote parts of the manuscript. B.M. co-supervised the project and wrote most of the manuscript.C.A.L. designed the study, co-supervised the project, and drafted parts of the manuscript. J.S. implemented the software and did most of the program evaluation. S.D. contributed to the program evaluation. M.G. and C.B. analyzed and discussed the results on GIGA-D-18-00169.pdfClick here for additional data file.GIGA-D-18-00169_R1.pdfClick here for additional data file.GIGA-D-18-00169_R2.pdfClick here for additional data file.Response_to_Reviewer_Comments_Original_Submission.pdfClick here for additional data file.Response_to_Reviewer_Comments_Revision_1.pdfClick here for additional data file.Response_to_Reviewer_Comments_Revision_2.pdfClick here for additional data file.Reviewer_1_Report_ -- Sharma Thankachan5/31/2018 ReviewedClick here for additional data file.Reviewer_3_Report_ -- Se-Ran Jun6/18/2018 ReviewedClick here for additional data file.Reviewer_3_Report_revision_1 -- Se-Ran Jun9/17/2018 ReviewedClick here for additional data file.Reviewer_3_Report_Revision_2 -- Se-Ran Jun9/29/2018 ReviewedClick here for additional data file.Reviewer_4_Report_ -- Sriram Chockalingam6/20/2018 ReviewedClick here for additional data file.Reviewer_4_Report_Revision_1 -- Sriram Chockalingam9/18/2018 ReviewedClick here for additional data file.Reviewer_5_Report_ -- Alexey Kozlov6/21/2018 ReviewedClick here for additional data file.Reviewer_5_Report_Revision_1 -- Alexey Kozlov9/18/2018 ReviewedClick here for additional data file.Supplemental FileClick here for additional data file.


We present a performance per watt analysis of CUDAlign 4.0, a parallel strategy to obtain the optimal pairwise alignment of huge DNA sequences in multi-GPU platforms using the exact Smith-Waterman method.Our study includes acceleration factors, performance, scalability, power efficiency and energy costs. We also quantify the influence of the contents of the compared sequences, identify potential scenarios for energy savings on speculative executions, and calculate performance and energy usage differences among distinct GPU generations and models. For a sequence alignment on chromosome-wide scale (around 2 Petacells), we are able to reduce execution times from 9.5 h on a Kepler GPU to just 2.5 h on a Pascal counterpart, with energy costs cut by 60%.We find GPUs to be an order of magnitude ahead in performance per watt compared to Xeon Phis. Finally, versus typical low-power devices like FPGAs, GPUs keep similar GFLOPS/w ratios in 2017 on a five times faster execution. Consequently, the focus of Genome Projects has moved from data production to data analysis and the central challenge nowadays is how to analyze such a huge amount of data in a rapid and accurate way. In order to face this challenge, biology and computer science joined forces, exploring solutions that demand sophisticated algorithms and powerful computing devices. As a result, refined solutions have been devised to support several biological subdomains such as protein structure prediction and docking . Fianlly, we use Si[1..n] to characterize a prefix with n characters, from the beginning of the sequence.In the first phase, the input sequences are Gx, where x is the size of the non-empty subsequence and G is the gap penalty. This represents the cost of aligning a non-empty subsequence with an empty one. In other words, the first row and column of the DP matrix are initialized with values Hj0,=−Gj and Hi,0=−Gi. Additionally, H0,0=0. The remaining elements of H are calculated with the recurrence relation Eq. Hm,n. In the initialization step, the first row and column are set to −p is usually the match value (ma) if S0[i]=S1[j] or the mismatch penalty (−mi), otherwise. If protein sequences are compared, p is given by a scoring matrix (e.g. PAM or BLOSUM).In Eq. S0 = ATTGTCAGGAGG and S1 = ACTTGTCCGAGA. The arrows indicate the cell from where the value was obtained, according to Eq. Hi−1,j−1, Hi,j−1, Hi−1,j).Figure Hm,n) and follows the arrows until cell H0,0 is reached. A left arrow in Hi,j .Note that many optimal global alignments may exist, since many arrows may be present in the same cell Local alignment must be employed when the goal is to obtain the similarity between regions inside the sequences. Smith and Waterman (SW)  proposedIn the initialization step, all elements of the first row and column are set to zero in SW. This is done because gaps should not receive any penalty at the beginning of the alignment.The second difference is the recurrence relation itself since since negative values are not allowed in SW, as shown in Eq. obtain the optimal local alignment), the algorithm starts from the cell which has the highest value in the DP matrix, following the arrows until a zero-valued cell is reached.In the second phase  than to extending it (Gextend). The cost of a sequence of gaps of length x in the affine gap model is γ(x)=Gopen+(x−1)∗Gextend.Algorithms NW “” sectionH, E and F, where H keeps track of matches/mismatches and E and F keep track of gaps in each sequence. Time and space complexities are quadratic. Matrix H is calculated with Eq. E and F use Eqs. Hm+1,n+1. To do the traceback, the arrows are followed in matrix H when a match/mismatch occurs or in matrices E and F, to track multiple gaps  to S1[1..j], DD(j) be the minimum cost of a conversion of S0[1..i∗] to S1[1..j] that ends with a gap, RR(n−j) be the minimum cost of a conversion of S0[i∗..m] to S1[j..n] and SS(n−j) be the minimum cost of a conversion of S0[i∗..m] to S1[j..n] that begins with a gap.Myers and Miller  (MM) adamatching procedure against a) vectors CC with RR and b) vectors DD with SS. The midpoint is the coordinate , where j∗ is the position that satisfies the maximum value in Eq. To find the midpoint of the alignment, the algorithm realizes a As in Hirshbergś, after the midpoint is found, the matrix is recursively split into smaller submatrix, until trivial solutions are found.Usually, a given protein sequence is compared against thousands or even millions of sequences that compose genomic databases. Also, two long DNA sequences with more than a million base pairs are often compared.In these scenarios, the use of exact algorithms such as NW and SW is often prohibitive in terms of execution time. For this reason, faster heuristic methods for local alignment were proposed which do not guarantee that the optimal result will be produced. Usually, these heuristic methods are evaluated using the concepts of sensitivity and sensibility. Sensitivity is the ability to recognize as many significant alignments as possible, including distantly related sequences. Selectivity is the ability to narrow the search in order to discard false positives . TypicalThe FASTA (FAST-All) algorithm  was propw in common and it is divided into three well-defined phases: seeding, extension and evaluation.BLAST   was propThe original BLAST algorithm searched for local alignments without considering gaps. In 1996 and 1997, two improved gapped versions of the original BLAST, NCBI-BLAST2  and WU-BThe Smith-Waterman (SW) algorithm has become very popular over the last decade to calculate the optimal pairwise comparison of (1) two DNA/RNA sequences or (2) a protein sequence (query) to a genomic database which is composed of several sequences.Both scenarios have been parallelized in the literature , 27, butStudies that deal with energy consumption are becoming relevant in DNA sequence comparison applications, which motivates to provide methodologies to measure energy in this context.Cheah et al.  apply anHasan and Zafar  present Zou et al.  analyze Benkrid et al.  perform Hi,j depends on three other cells: Hi−1,j, Hi−1,j−1 and Hi,j−1. The parallelization of this kind of dependency is traditionally made by the wavefront method [SW executes in two phases: (1) compute the DP matrix and 2) traceback. Most of the time is spent in phase 1 and, therefore, this phase is often parallelized. Equation  tracebact method , in whicd1. In step 2, both cells of diagonal d2 are computed in parallel. In steps 3 to 5, the number of cells that can be calculated in parallel increases until it reaches the maximum parallelism in diagonal d5. The maximum parallelism is kept in the computation of diagonals d5, d6, d7, d8 and d9 . In the computation of diagonals d10 to d12, the parallelism decreases until a single cell is computed in diagonal d13. In the wavefront method, parallelism is non-uniform and it is explored in a limited way when the wavefront is being filled (beginning of the computation) and when it is being emptied (end of the computation).The wavefront method is illustrated in Fig. CUDAlign  obtains When CUDAlign is executed in a platform composed of multiple GPUs, a challenging scenario arrises. The computation of the SW matrix is split among the GPUs and each GPU calculates a subset of columns. GPUs are arranged logically in a linear way, sending the border column elements to the next GPU. Communication between neighbor GPUs is overlapped with computations, i.e., it is carried by asynchronous CPU threads while the GPUs keep computing Fig. .Fig. 5TCUDAlign 4.0  introducThe IST strategy for stage 2 was built based on PT strategy. IST uses the knowledge obtained in several analyses, particularly that the best score in border columns tend to coincide with the crosspoints in them. With this in mind, IST speculates crosspoints in intermediate border columns and Stage 2 in Fig. CUDAlign was implemented in CUDA, C++ and pthreads. Results obtained in a large GPU cluster using long DNA sequences present good scalability for up to 16 GPUs . Using CTable We have conducted an experimental study on a computer endowed with an Intel Xeon server, where we have plugged different number and models of Nvidia GPUs (they will be changing depending of the experiment performed). See Table As input, we used real DNA sequences obtained from the National Center for Biotechnology (NCBI) . We compThe execution of the required stages on a multi-GPU platform required two modifications in CUDAlign 4.0: 1) partitioning the input sequences to fit them in texture memory, and (2) saving extra rows to the file system as marks to be used in later stages to find crosspoints with the optimal alignment. Furthermore, our experimental analysis is done on the first three stages of CUDAlign, which are the ones executed on GPUs as shown in Table  partitioOur system to measure current, voltage and wattage was built based on a Beaglebone Black, an open-source hardware  combinedThe Accelpower module uses a modified version of pmlib library , a softwOur procedure for measuring energy begins with a start-up of the server daemon. Then, the CUDAlign 4.0 source code was modified in order to include the measurement calls within the GPU code as shown in Fig. The input data volume.The computational stage within SW.The multi-GPU factor.The cost of speculative executions.Comparison among GPUs (different generations and models).Comparison with other accelerators (Xeon Phis) and low-power devices (FPGAs).We study performance and power efficiency on GPUs from different perspectives, studying the influence of a wide variety of issues, namely: Now we dedicate a different subsection to analyze each of them separately.We have executed the modified CUDAlign 4.0 version using four different comparisons  during the time the GPUs are otherwise idle, and (2) showing a very high speculation hit ratio. In terms of power consumption, the first premise consumes extra energy, which would only be amortized through a much shorter execution. And mispredictions may also jeopardize the GFLOPS/w ratio.In order to analyze energy costs and establish amortization times for CUDAlign 4.0 in multi-GPU environments, we have monitored power consumption at real-time on a dual GPU execution performed on Titan Maxwell GPUs. Figure Table Our next analysis compares different GPU generations and models. We have GPUs coming from three different generations , and from two different budgets (two mid-end GTXs and two high-end Titans). Figure Stage 1 reduces power in its final part on Titan GPUs.Stage 2 has a different pattern on every GPU, predominating regions of lower power on newer GPUs, particularly at the end of the process.Stage 3 shows the same pattern in all cases, and average power is higher on newer GPUs, but this is caused by a much higher throughput when computing and bandwidth when communicating, leading to power savings overall.Going stage by stage, we can see that: Table Concerning power efficiency, GTX models behave better than Titans. And we are nicely surprised by energy costs: in less than five years, we have been able to reduce the energy cost of running our algorithm by as much as 60%.For Megabase DNA sequences, the SW matrix is several Petabytes long, and so, few implementations allow alignments for DNA sequences longer than 10 MBP (Million Base Pairs) like CUDAlign does.SW#  performeMore recently, Rucci et al.  have devTable To dedicate few words to CPUs, Korpar et al.  already FPGAs, on the other hand, are much slower than GPUs on large DNA sequences, but tough competitors regarding power efficiency. In fact, we compiled in “Along this paper, we have studied GPU acceleration and power consumption on a multi-GPU environment for the Smith-Waterman method to compute, via CUDAlign 4.0, the biological sequence alignment for a set of real genome scale DNA sequences coming from human and chimpanzee homologous chromosomes retrieved from the National Center for Biotechnology Information (NCBI).We may distinguish 6 stages within CUDAlign 4.0, and the first half have been implemented in CUDA for its acceleration on GPUs. On a stage by stage analysis, the first one is more demanding and takes the bulk of the computational time. On the other hand, power consumption was kept more stable across executions of different alignment sequences, though it suffered deviations of up to 30% across different stages.One of the major innovations in CUDAlign 4.0 was the Incremental Speculative Traceback (IST)  introducIn a multi-GPU environment composed of 4 GTX980 GPUs, we reduce the computational time by half when compared to a dual GPU execution, at the expense of just 3% penalty in power usage. That way, our experiments demonstrated an efficient correlation between acceleration and extra energy required.Overall, we have reduced execution times from 9.5 h on a Kepler GPU to just 2.5 h on Titan Pascal, with energy costs cut by 60%. Compared to FPGAs, which have an excellent reputation as low-power devices, GPUs are competitive and keep similar GFLOPS/w ratios in 2017 while maintaining the leadership as HPC accelerators for a five times faster execution.We expect GPUs to increase their role as high performance and low power devices for biomedical applications in future GPU generations, particularly after the introduction in early 2017 of the 3D memory within Pascal models.


In general, it is difficult to identify previously uncharacterised cis-acting elements with an unknown consensus sequence. The task is especially problematic with viruses containing regions of limited or no similarity to other previously characterised sequences. Fortunately, the fast increase in the number of sequenced genomes allows us to detect some of these elusive cis-elements. In this work, we introduce a web-based tool called cRegions. It was developed to identify regions within a protein-coding sequence where the conservation in the amino acid sequence is caused by the conservation in the nucleotide sequence. The cRegion can be the first step in discovering novel cis-acting sequences from diverged protein-coding genes. The results can be used as a basis for future experimental analysis. We applied cRegions on the non-structural and structural polyproteins of alphaviruses as an example and successfully detected all known cis-acting elements. In this publication and in previous work, we have shown that cRegions is able to detect a wide variety of functional elements in DNA and RNA viruses. These functional elements include splice sites, stem-loops, overlapping reading frames, internal promoters, ribosome frameshifting signals and other embedded elements with yet unknown function. The cRegions web tool is available at  Mostly, the amino acid sequence of a protein is conserved in order to maintain its function and structure. However, the conservation may also be caused by the selection at the nucleic acid level due to essential cis-acting sequences located in the protein-coding region. Thus, certain regions in a protein-coding sequence might encode specific amino acids not because of the selective pressure to the amino acid sequence, but because of the conservation at the nucleic acid level in DNA or RNA. There can be multiple reasons: the existence of nucleic acid secondary structures, splice sites, binding sites for proteins (e.g. transcription factors) or short RNAs, internal promoters, ribosome frameshifting signals, subgenomic promoters in RNA viruses, viral packaging signals and other regulatory elements. Additionally, the conservation at nucleic acid level might exist due to overlapping reading frames, which are common in viruses but also occur in cellular organisms .Computational annotation is extremely important when the experimental annotation is impracticable, for example, in case of organism or hosts which are uncultivable. However, thanks to the massive deployment of second-generation sequencing, the number of complete or near-complete genomes of previously unknown viruses has increased tremendously. This is one of the reasons why comparative analysis and computational annotation is needed to get some insight into the molecular biology of these viruses. Additionally, it has been shown that a large number of these new viruses will most likely constitute a new viral genus or even a family . SometimSeveral studies have used synonymous substitution restriction to identify overlapping or embedded functional elements in coding sequences of viruses . Howeverhttps://github.com/bioinfo-ut/cRegions. The workflow of cRegions is as follows:Two inputs are required: a protein multiple sequence alignment (MSA) and nucleic acid sequences containing coding sequences (CDS) of respective proteins. Both inputs have to be in FASTA format. mRNA or the full genome can be used instead of the exact CDS. However, the coding sequence must not contain introns.Protein alignment is converted into a corresponding codon alignment using respective coding sequences with PAL2NAL .Henikoff position-based sequence weights are calculated using the codon alignment .Codon usage bias is calculated from the codon alignment. Calculated proportions are adjusted by Henikoff position-based sequence weights to account for non-uniform phylogenetic coverage . Codon pExpected nucleotide proportions are calculated for each position in the codon alignment based on the amino acid sequence and the codon usage bias . Henikoff position-based sequence weights are used to adjust expected proportions .p-values of Chi-square goodness of fit test (chisq.test) for each column in the codon alignment. The test allows us to see whether the observed distribution of nucleotides is significantly different from expected distribution. We use the negative logarithm of the p-value of the Chi-square goodness of fit test as the metric. Bonferroni correction is used to show the threshold with significance level α = 0.05.The algorithm uses R  to calcuThe second metric is the root-mean-square deviation (RMSD). Only nucleotides which have a predicted probability over zero are included in the RMSD calculation.The third metric is the maximum difference (MAXDIF), which selects only a single nucleotide from each column having the highest absolute difference between predicted and observed values.The subscript ‘exp’ indicates expected frequencies.The observed nucleotide frequencies are compared to the expected probability distribution of nucleotides in each position. The comparison is made only for positions with amino acids having more than one codon. Three different metrics are used for this:The overall principle of the cRegions tool is to compare observed nucleotide frequencies to expected probability distribution and calculate appropriate metrics (described below) to detect regions where the coding sequence is more conserved than expected. Scripts used in the cRegions web tool are available in GitHub repository at In all cases, the larger numerical value of a metric indicates higher conservation at the nucleic acid level. Additionally, if a position in the codon alignment has more than 20% of gaps, the metric is not calculated for that position .In the present work, we used the non-structural and structural polyprotein of alphaviruses as an example. Sequences were downloaded from the NCBI viral genome database . The first dataset consists of 24 known alphaviruses . The nonhttp://www.ebi.ac.uk/Tools/msa/mafft/. Graphs were created with default settings using a sliding window with size one unless stated otherwise. Codon alignments for the synplot2 (n + 1 codons). In case of synplot2, significant hits were selected at threshold p < 10−5. It is less conservative compared to the threshold used in the synplot2 paper . Alignment visualisation is provided by MSAViewer . A total of 11 entries were removed as they did not have annotated full-length non-structural polyprotein. Identical protein sequences were removed using jalview ‘remove redundancy 100’ (NC_001449).The Venezuelan equine encephalitis virus (VEEV) neighbour sequences were downloaded from the NCBI viral genomes database (ncy 100’ . The finhttp://www.bioinformatics.org/sms2/random_coding_dna.html). A different number of random mutations  were introduced into that sequence with the SMS2 mutate tool (http://www.bioinformatics.org/sms2/mutate_dna.html)  . In totani = the average pairwise identity of nucleotide sequencescRegion algorithm assumes that, in general, the distribution of nucleotides at each position in the MSA correspond to an average codon usage of the same sequences under analysis. The assumption is reasonable if the sequences under analysis have diverged. However, in the case of nearly identical sequences, the distribution of nucleotides in each position in the MSA is more similar to observed nucleotide proportions rather than to average codon usage. Therefore, the expected proportions are more similar to observed proportions. Thus, even if using Bonferroni correction, the Chi-square test may still give many potentially false positive signals. Therefore, we need to adjust the threshold such a way that in the case of nearly identical sequences the threshold is stricter. For that, we also include the average pairwise identity of nucleotide sequences into calculations of expected nucleotide proportions at each position. The expected nucleotide proportions are adjusted by the observed proportions which depend on the average pairwise identity of nucleotide sequences. The exponent value was found empirically.In addition to the previous adjustment of expected values, also the threshold itself is adjusted. The threshold correction depends on the ratio between the average pairwise identity of nucleotide and protein sequences and the number of sequences in the MSA.t = threshold − log10ni = the average pairwise identity of nucleotide sequencespi = the average pairwise identity of protein sequencesd = the number of sequences in the multiple sequence alignmentFirst, we applied cRegions and synplot2 on all 24 non-structural polyproteins of alphaviruses . We detected a total of six significant signals with cRegions  and threThe second significant hit, a single nucleotide at position 1,149, was detected only by the cRegions algorithm. However, two adjacent positions 1,143 and 1,146 were just below the threshold. In the New World alphavirus dataset, in addition to position 1,149, 1,143 was also significant. The signal is just adjacent to the packaging signal of SINV and New World alphaviruses . It has been shown that a 570 nt fragment positions 684–1,253 from the SINV binds to the viral capsid protein and is required for packaging of SINV. The detected signal lies in this region . HoweverThe third and the fourth signal are also single nucleotides at positions 2,835 and 2,967, respectively . Both siThe fifth significant hit is a single nucleotide at position 6,834. It is downstream of the ‘leaky’ stop codon . Synplot2 was able to detect a much larger region compared to cRegions in the same area  and S5A.The sixth signal consists of two positions 8,658 and 8,664 on the codon alignment  and S5A.The cRegions and the synplot2 were also applied to the structural polyproteins of alphaviruses . Sliding window size 2 was used with cRegions. A strong signal was detected in positions 2,643–2,649 on the codon alignment, which corresponds to a UUUUUUA motif . The motThe method used in cRegions has some limitations and prerequisites . First, As cRegions was designed to work on diverged sequences, the method may give potential false positive signals in low divergence sequences or in sequences locating close to neutrally evolving sequences ; Fig. S8Given sufficient evolutionary time, the conservation of amino acid residues in different homologous sequences will not necessarily imply the same conservation at nucleotide level due to the redundancy of the genetic code. However, it can be reasoned that functionally important cis-acting elements embedded in protein-coding sequences will be evolutionarily conserved, even if these regions are subject to constant evolutionary pressure both through their translation product (amino acid sequence) and cis-acting functions.Previous studies have produced numerous valuable methods for detecting overlapping or embedded functional elements in coding sequences, mainly by identifying regions of excess synonymous constraints . CurrentDuring evolution, homologous protein-coding sequences accumulate random substitutions by switching to different synonymous and non-synonymous codons. In addition, if purifying selection acts on a protein sequence, synonymous substitutions are favoured over non-synonymous substitutions. However, synonymous codons are not used with equal proportions . One hypIn addition, host adaptation theory (tissue adaptation) may explain the preferential use of synonymous codons. It has been shown that the codon usage is strongly related to the specific host in both bacterial and human viruses. Also, the highest level of adaptation to host codon usage is for proteins which appear abundantly in the virion, meaning that the codon usage of virion and non-virion proteins differ . AdditioHowever, authors of the synplot2 used neutral evolution as the null model, therefore not including codon usage bias into the analysis. They reasoned that it was not required based on the results and it would be impossible to accurately estimate given the limited genome size of RNA viruses . Also, SCalculating the preferential use of synonymous codons may result in incorrect assessment on many occasions. First, if the analysis is performed on sequences with low divergence, then the estimation of codon usage may be biased. Second, a large overlapping region or the abundance of rare codons in a gene may also affect codon usage estimation. The same conclusion was reached in the synplot2 paper by Firth, where he noted that the divergence parameters of the null model are determined from the full coding region and if the alignment contains extensive overlapping regions then the neutral divergence rates will be underestimated . As a reHowever, not all synonymous codons should be treated as equals. In case of serine, two mutations are necessary to move from AG[A/G] serine codon block to TCN codon block (in the standard codon table). Therefore, usage for codons in the TCN block and in the AG[A/G] are calculated separately. For example, if only AG[A/G] serine was observed, then only AG[A/G] codon proportions were used for predictions and vice versa. This is not implemented in the method published by As it is impossible to assess the conservation at the nucleic acid level if an amino acid is encoded only by a single codon (e.g. methionine and tryptophan), therefore these amino acids are excluded from the analysis. For these positions, it is unknown if the conservation is due to an amino acid or DNA/RNA constraint or by pure chance. cRegions will not calculate metrics for these positions. In the work done by Databases often contain a redundant set of sequences. Therefore, it is very important to include phylogenetic weighting in the analysis. A redundant and biased dataset or a dataset with low variability will affect predictions and therefore may cause false positive signals. Henikoff position-based sequence weighting was used to compensate for the over-representation of similar sequences or taxa in the codon alignment . Tree-ban = 1). However, cis-acting sequences shorter than three codons  may be masked by adjacent low scoring areas. Thus, in contrast to them, cRegions allows also single-codon resolution. We would like to note that synplot2 provides single-codon scores in an output text file and these can be used for analysis of regions shorter than three codons.Cis-acting elements may be longer than a single codon, for example, dual-coding regions, thus the possibility to calculate a single metric over consecutive codons may be preferred. Sliding window approach is used in synplot2, FRESCo, and method described by In this study, we applied cRegions to the non-structural and structural polyprotein of alphaviruses as an example. The final dataset contained 24 sequences . The diversity and the number of sequences were sufficient in our analysis to detect the majority of known cis-acting elements in alphaviruses.Several alphaviruses contain an in-frame termination codon and use termination read-through to produce the p1234 non-structural polyprotein . DatasetBoth programs: cRegions and synplot2 are able to detect a signal, even if the alignment is not perfect. In the codon alignment of the structural polyprotein, UUUUUUA motif was misaligned in two sequences (SPDV and SDV).Protein-coding sequences must have diverged. Thus, sufficient evolutionary time is needed for substitutions to occur in homologous genes in different species/isolates.Only those embedded element in a coding-sequence can be detected which are or have been under selection.This method is not applicable to neutrally evolving genes. In the case of neutrally evolving genes, we recommend using synplot2 or other similar solutions.Cis-acting sequences must be conserved in respect to amino acid sequences.It is impossible to assess conservation at the nucleic acid level if an amino acid is encoded by a single codon (e.g. methionine and tryptophan).Long dual-coding areas or abundant rare codons will affect codon usage estimation.A low number of sequences may reduce the signal to noise ratio.Bad alignment quality, especially near large gaps may affect the results. Therefore, usage of different alignment methods or manual correction of the alignment is recommended.We have shown that cRegions is capable of detecting different cis-elements. However, the method has multiple prerequisites and limitations:We would like to note that cRegions is not restricted to viral genes, but the method can be applied to any set of diverse set of protein-coding sequence if the prerequisites are fulfilled.Depending on the number and divergence of the protein and nucleic acid sequences, the size of the region and other conditions, different approaches like the synplot2, FRESCo or the method developed by Evolutionary conserved embedded functional elements within an open reading frame are often overlooked as they are difficult to detect without specialised bioinformatics tools . In thishttp://bioinfo.ut.ee/cRegions/ and source code is available in GitHub repository at https://github.com/bioinfo-ut/cRegions.Web tools like the cRegions and the synplot2 are important for finding functional embedded elements in coding sequences and are easy to use for non-bioinformaticians. The cRegions web tool is available at 10.7717/peerj.6176/supp-1Supplemental Information 1‘New World’ alphaviruses are marked with asterisk and ‘SFV Complex’ alphaviruses are written in bold.Click here for additional data file.10.7717/peerj.6176/supp-2Supplemental Information 2(A) Henikoff position-based sequence weights are calculated for each sequence based on the codon alignment. (B) Glutamic acid is encoded by two codons, therefore, the observed proportion for GAA is 0.625 and for GAG 0.375. (C) Henikoff position-based sequence weights are used to compensate for the over-representation of well-sequenced taxa in the multiple sequence alignment. The proportion for GAA and GAG are based on sequence weights.Click here for additional data file.10.7717/peerj.6176/supp-3Supplemental Information 3(A) Example dataset consists of 7 sequences. Five sequences are very similar; therefore, they have a low weight. Two sequences out of seven are different, therefore, having a high weight. (B) Glutamic acid is encoded by only two codons: GAA and GAG. Only the third position of a codon varies, therefore, including information. The observed proportion for the A nucleotide in the 3rd position of the glutamic acid codon is 0.714 and for G it is 0.286. (C) Predictions based on codon usage give us proportion for A is 0.55 and for G 0.45. By comparing observed and predicted proportions we will get a signal as there is a difference. However, it is a false positive signal due to a biased dataset. (D) Adjusting predictions with sequence weights, we can account for the over-representation of similar sequences. Only nucleotides that were observed are adjusted.Click here for additional data file.10.7717/peerj.6176/supp-4Supplemental Information 4Allowed gaps parameter is a threshold (percentage of gaps in one column) if a metrics  should be calculated to a certain position. Metrics are not calculated at positions (red crosses) where the percentage of gaps in one column exceeds the threshold . By default, a position (column) must have less than 20% of gaps. Skip gaps parameter is only used in sliding window mode. It is a threshold for skipping columns during sliding window mode instead of terminating the calculation as in the first three positions (NA). The threshold is used to avoid sliding window calculation termination while encountering insertions in a few sequences. By default, if the proportion of gaps in a position exceeds 90%  then this position is skipped (transparent red column) and next position is included to the window.Click here for additional data file.10.7717/peerj.6176/supp-5Supplemental Information 5http://www.ebi.ac.uk/Tools/msa/mafft/. Codon alignment was generated with pal2nal .(A) Non-structural polyprotein alignment of all 24 Alphaviruses in our dataset. (B) Non-structural polyproteins of ‘New world’ Alphaviruses. (C) Non-structural polyproteins of ‘SFV Complex’ Alphaviruses. Non-structural polyprotein sequences were aligned with MAFFT using the default settings at Click here for additional data file.10.7717/peerj.6176/supp-6Supplemental Information 6http://www.ebi.ac.uk/Tools/msa/mafft/. Codon alignment was generated with pal2nal .Same settings were used in the synplot2 publication . (A) NonClick here for additional data file.10.7717/peerj.6176/supp-7Supplemental Information 7http://www.ebi.ac.uk/Tools/msa/mafft/. Codon alignment was generated with pal2nal .Significant signal was detected between codons 800–1000, which corresponds to a known UUUUUUA motif. The y-axis on the upper part of the synplot2 figure shows the p-value and the lower part shows obs/exp rato. Structural polyprotein sequences were aligned with MAFFT using the default settings at Click here for additional data file.10.7717/peerj.6176/supp-8Supplemental Information 8http://www.ebi.ac.uk/Tools/msa/mafft/. Codon alignment was generated with pal2nal .(A) Two known regions of RNA secondary structures were detected with cRegions using sliding window size 18. (B) Zoomed region of the conserved stem-loops. Locations of the signals in the VEEV genome are also provided . (C) Zoomed region of the stem-loops adjacent to stop codon. Locations of the signals in the VEEV genome are also provided . (D) SimClick here for additional data file.10.7717/peerj.6176/supp-9Supplemental Information 9http://www.bioinformatics.org/sms2/random_coding_dna.html) (http://www.bioinformatics.org/sms2/mutate_dna.html) (A random 3000nt long protein-coding sequence was created with SMS v2 tool (na.html) . A diffena.html) . Each siClick here for additional data file.