Motivation: Metagenomics characterizes microbial communities by random shotgun sequencing of DNA isolated directly from an environment of interest. An essential step in computational metagenome analysis is taxonomic sequence assignment, which allows identifying the sequenced community members and reconstructing taxonomic bins with sequence data for the individual taxa. For the massive datasets generated by next-generation sequencing technologies, this cannot be performed with de-novo phylogenetic inference methods. We describe an algorithm and the accompanying software, taxator-tk, which performs taxonomic sequence assignment by fast approximate determination of evolutionary neighbors from sequence similarities.Results:Taxator-tk was precise in its taxonomic assignment across all ranks and taxa for a range of evolutionary distances and for short as well as for long sequences. In addition to the taxonomic binning of metagenomes, it is well suited for profiling microbial communities from metagenome samples because it identifies bacterial, archaeal and eukaryotic community members without being affected by varying primer binding strengths, as in marker gene amplification, or copy number variations of marker genes across different taxa. Taxator-tk has an efficient, parallelized implementation that allows the assignment of 6 Gb of sequence data per day on a standard multiprocessor system with 10 CPU cores and microbial RefSeq as the genomic reference data.Availability and implementation: Taxator-tk source and binary program files are publicly available at http://algbio.cs.uni-duesseldorf.de/software/.Contact:Alice.McHardy@uni-duesseldorf.deSupplementary information:Supplementary data are available at Bioinformatics online. Metagenomics allows us to study microbial communities from natural environments without the need to obtain pure cultures of the individual member species  data.From a methodological standpoint, the differences between the phylogenetic-placement-based methods for profiling and alignment-score-based methods for taxonomic binning and profiling, such as  (CARMA3  or SOrT-taxator-tk) is a software package for the taxonomic sequence assignment in shotgun metagenomics with applications to both profiling and binning. Conceptually, it lies between sequence similarity-based programs which use local sequence alignment scores and those using trees. Taxator-tk extends the alignment score-based approach by approximating phylogenetic gene trees and thereby provides more accurate taxonomic assignments, without assuming universal, rank or clade-specific gene conservation levels as parameters. We improve in terms of applicability to large datasets compared with phylogenetic methods by assigning genomic sequences without the computationally demanding steps of de-novo multiple sequence alignment (MSA) and tree inference. Taxator-tk determines a subset of homologs, which represent the approximate evolutionary neighbors for a query sequence, by a linear number of pairwise sequence comparisons with regard to the number of considered homologs and then assigns a taxon ID using a reference taxonomy based on the taxonomic IDs of these neighbors. We have furthermore reduced the run-time by limiting the analysis to distinct homology-supported regions of the query sequence, which we termed query segmentation. Our open-source (GPLv3) software can be applied to arbitrary nucleotide sequences, such as reads, contigs, scaffolds and complete genomes sequences. It can be downloaded from http://algbio.cs.uni-duesseldorf.de/software/.Our taxator toolkit (RefSeq (mRefSeq) . At the beginning of the taxator algorithm in Stage 2, overlapping regions on the query, each defined by local alignment to a nucleotide reference sequence, are merged into larger subsequences called segments . These query segments are flanked by regions without similarity to any reference data  and are not considered further. This step reduces the overall number of positions in the following alignment computations and improves the taxonomic assignment of queries that have undergone genome rearrangements, resulting in a different order of these segments. The reference sequence regions corresponding to the local alignments are extended at both sides by the missing number of nucleotides to match to the corresponding query segment with respect to its length and we refer to these as reference segments. Each independent set of homologous segments is the input to the core algorithm in the program taxator in Stage 2  of weighted assignments with a minimum number of identical bases (default = 50 bp) . Binner has the optional parameters minimum sequence identity and minimum sample abundance, but these were not applied in our analysis. If the taxonomic information is limited or contradictory, taxator and binner assign identifiers to higher ranking taxa in a conservative fashion to obtain the most reliable taxonomic assignments.In the third stage c, multipq of the original query sequence from an (unknown) taxon Q and a set of homologous segments with known taxon IDs. The term ‘segment’ refers to a gap-less subsequence of either the query or a reference sequence. Given that for the set of homologs we know the correct underlying species tree of taxa (q).The input to the algorithm is a segment  of taxa a, we can of taxa a. If we taxator algorithm attempts to identify these segments with a linear number of pairwise segment comparisons. Let us consider an undirected graph in which nodes represent the segments (tree leaves) and edge lengths the evolutionary distances between pairs of segments within the underlying tree  are added to M  is larger than distance. In the second pass, all segments are then aligned to o and segment taxa with distances smaller than or equal to distance are added to M as well  to determine a meaningful taxon ID. The taxa in M become more diverse if the alignment scores are inaccurate ultrametric distance estimates, if the species subtree’s topology deviates from the respective part of the taxonomy or if the gene tree’s topology deviates from the species tree, for instance due to varying rates of evolution or the inclusion of non-homologous segments in the analysis. The robustness of the algorithm in avoiding incorrect assignments under these circumstances relies on the number of taxa in M and the subsequent LCA operation. Further details relating to the robustness of the implementation are given in the Supplementary Materials, ‘I. Taxonomic assignment of sequence segments’.If no outgroup could be determined or if Supplementary Materials, ‘Performance measures’). The macro-precision specifies the fraction of correct assignments per predicted bin (precision), averaged over all such bins, whereas the macro-recall measures the fraction of correctly recovered sequence data per truly existing bin , averaged over all such bins. To account for strong differences in bin size, we also pooled the species, genus and family assignments, and reported the overall precision for these ranks as the total fraction of correct assignments. We tested the assignment performance of different methods using three simulated short read datasets, simulated 16S rRNA data, three simulated metagenome contig datasets and using assembled cow rumen metagenome contigs. For every simulated dataset, we performed seven cross-validation experiments . In each experiment, we simulated a specific minimum taxonomic distance between a query sequence and the reference sequences. For the first experiment, all reference data, including the species genome data from which the query had been sampled, were made available to the method for assigning a single query sequence as an idealized test case. In the other six scenarios, all reference data belonging to the species, genus, family, order, class or phylum of the query sequence, respectively, were made inaccessible for the method in leave-one-taxon-out cross-validation experiments. We summarized the sequence assignments from these experiments to characterize a method’s assignment performance across the entire range of taxonomic distances. For evaluation with the cow rumen metagenome sample, for which no true taxonomic labels were known, we divided the assembled contigs into multiple sequence ‘chunks’ and characterized the consistency of taxonomic assignments for chunks originating from same contig .Before evaluating any method, we removed the smallest predicted bins (1%) as likely errors. We used the macro-precision and macro-recall as measures of assignment performance . This served as a proof of concept, as taxator-tk classifies arbitrary sequence regions including taxonomic marker genes. We did not expect it to perform better than sophisticated phylogenetic models for this task, but wanted to confirm a satisfactory performance. The macro-precision for the taxonomic assignment of 7176 16S rRNA genes  was constantly above 92%  in the combined cross-validation (Methods), using the whole-genome reference sequences in mRefSeq47 , not just the 16S genes. More precisely, the average error rate per bin (one minus precision) was 7.4% at the species level and 4.6% at the order level.We first evaluated the performance of mRefSeq47 and evaluated taxator-tk with these three datasets using the (combined) cross-validation experiments. The performance was very similar for the different fragment sizes . Overall, taxator-tk showed high precision in simulated read assignment: the macro-precision for all short read lengths remained above 74% and was 82–99% for the genus to kingdom ranks, about 10% lower on average than for the 16S data. This was still good for the assignment of short sequence fragments from arbitrary genomic regions compared with a marker gene. At genus level, the macro-recall was 19–23% (∼33% genera recovered) if genome sequences of the same species as the query sequence were provided in the reference  and as low as 5–7% (∼16% genera recovered) otherwise . The macro-recall depends on the availability of related reference data at the respective ranks. It decreases when removing reference data for cross-validation. For example, if all reference data at genus level are removed, then no correct assignments to the genus rank are possible. For lower taxonomic ranks, the macro-recall was also low due to the large number of sample taxa and their uneven representation caused by the taxonomic bias toward a few abundant phyla in mRefSeq47. The longer reads had a slightly higher macro-recall than the shorter ones. Since longer sequences yield better recall and because overlapping reads contain redundant information, leading to more alignment computations, we recommend applying taxator-tk to  assembled data. For longer query sequences, we were more likely to find segments for processing and therefore to assign a larger portion of the sample.Next, we simulated 100 000 reads at 100, 500 and 1000 bp by subsampling randomly from 1729 species in taxator-tk to CARMA3 and MEGAN4/5 using the same taxonomy and the same nucleotide alignments against mRefSeq54 . Additionally, we applied these three methods to two datasets using protein-level alignments which we inferred using BLAST+/tblastx. When doing so, we used the programs recommended parameter settings  and cross-validation, as before .For our tests on three simulated contig samples, we compared Supplementary Fig. S10) for our evaluation. This sample includes 49 equally abundant species (51 strains) and was created by Illumina paired read simulation with pIRS , taxator-tk produced substantially fewer errors: sequence assignments to species, genus and family were 91% correct for taxator-tk, compared with 52% for CARMA3 and 59% for MEGAN4. Accordingly, taxator-tk showed the highest macro-precision of all methods, e.g. 61% at the species level, compared with 3% (CARMA3) and 5% (MEGAN4). The low macro-precision observed for CARMA3 and MEGAN4 is largely due to the prediction of many small bins with many false assignments . The majority of assignments were to bacteria, archaea or undetermined in the case of CARMA3, because we restricted the availability of similar reference sequences in each of the individual cross-validations, which we then jointly assessed.We created a simulated NGS metagenome dataset , taxator-tk was also the most precise, though it had a lower recall than the other methods . Differences in assignment precision were also evident in the number of predicted taxon bins: for instance, when simulating novel families , many more species bins were predicted by CARMA3 (1672) and MEGAN4 (824) than by taxator-tk (65), with 49 species being present in the sample. Similarly, MEGAN4 predicted 69 orders, CARMA3 81 and taxator-tk 27, compared with the existing 32 orders in simArt49e  estimate of macro-precision . Overall, the bins predicted by taxator-tk were smaller, more precise and much more likely to represent truly existing taxa than those predicted by the other programs although larger bins tended to be more accurate for all methods. CARMA3 and MEGAN4 predicted a substantial number of mostly smaller-sized incorrect bins. Although the size-dependent kNN precision curves at large bin sizes is unaffected by these small bins, the curves remained below 70% (CARMA3) or 80% (MEGAN4), whereas the taxator-tk curve reached almost 100%. For the smallest bins, taxator-tk’s kNN precision was approximately 20% whereas bins below 500 kb for CARMA3 and MEGAN4 were practically indistinguishable from noise. This shows that the high macro-precision with taxator-tk is not only due to a lower frequency of falsely predicted bins, but also due to a substantially higher precision for the large bins.To investigate the reason for the observed differences between overall and macro-precision, which reflect variations in assignment precision for bins of different sizes, we plotted the per-bin precision at the family level in the combined cross-validation, as a function of predicted bin size with a taxator-tk had the highest macro-precision and the most realistic number of predicted taxon bins .Next, we performed cross-validation on the FAMeS . We alsoPhyloPythiaS . The performance and error distributions for the similarity-based programs  were consistent with our previous evaluations with SimMC. MEGAN4 and MEGAN5 produced almost identical results. Using protein local alignments, we observed a moderate increase in overall species to family precision for MEGAN5 and CARMA3, while taxator-tk improved in macro-recall. Notably, taxator-tk showed the best macro-precision of all similarity-based programs and all ranks, regardless of which alignment kind was used. Kraken produced most errors and the lowest macro-precision, because it assigned almost exclusively at species level. This would make it generally unsuitable in situations where sequences of closely related genomes are unavailable. However, it had a comparatively high macro-recall up to the order level.For the contig assignments of the composition-based program PhyloPythiaS showed that composition-based classification, when supplied with limited amounts of additional training data from the relevant species, correctly assigned most data at the genus and family levels , which were either rarely assigned by taxator-tk or mostly incorrectly assigned by CARMA3, MEGAN and Kraken. However, PhyloPythiaS predicted only 6 families compared with 29 underlying families, versus 43 (Kraken), 14/18 (taxator-tk), 50/32 (CARMA3) and 17/18 (MEGAN5) with nucleotide or protein alignments, respectively. PhyloPythiaS had the highest macro-recall. The macro-precision  was also higher than for Kraken (∼4–13%), MEGAN (∼7–31%) or CARMA3 (∼7–48%) but less than for taxator-tk (∼32–68%). However, unlike for the other programs, the modeled taxa for PhyloPythiaS should be specified a priori to achieve optimal performance. It is therefore best applied when the taxonomic composition of a microbial community has already been determined and sufficient training data are available for the identified taxa.Assignment with taxator-tk, CARMA3, MEGAN4/5 and PhyloPythiaS . We considered scaffolds to be less reliable than contigs, which we reconstructed by splitting the available scaffolds at gaps of more than 200 positions . We subsequently divided contigs longer than 10 kb into sequence ‘chunks’ of 2 kb, resulting in a 319-Mb dataset, which we used to assess the assignment consistency for chunks originating from the same contig. The chunk sequences were assigned with taxator-tk, CARMA3, MEGAN , Kraken and PhyloPythiaS. As the standard of truth for each contig, we determined the taxon minimizing the inconsistency between all corresponding chunk assignments : 76–89% macro-consistency at species-to-order level, in comparison to MEGAN (34–40%), CARMA (0–55%), Kraken (32–35%) and PhyloPythiaS (56–65%). The overall consistency  for species-to-family levels was 97%/97% with taxator-tk, 39%/48% with CARMA3, 62%/64% with MEGAN (nucleotide/protein-level), 42% with Kraken and 82% with PhyloPythiaS. Likewise, taxator-tk assigned less data at species-to-family level, with a total of 13/12 Mb being consistent compared with CARMA3 (8/26 Mb), MEGAN (42/47 Mb), Kraken (19 Mb) or PhyloPythiaS (14 Mb). The different methods again determined different numbers of taxa: CARMA3 predicted 572/611 genera with a macro-consistency of 53%/31%, MEGAN 264/203 genera (34%/37%), Kraken 661 (32%), PhyloPythiaS 33 (63%) and taxator-tk found 110/27 genera (76%/81%). The high consistency values observed for taxator-tk indicate that it is a precise taxonomic classifier for real metagenomic contigs.In agreement with the results for the simulated metagenome datasets, the BLAST or LAST. Thus, the run-time for the initial similarity search was identical for all methods. We determined the time for the taxonomic assignment of simArt49e for all methods when performing a cross-validation with families present in the test dataset removed from the reference data (Kraken (single CPU core and ∼100 GiB RAM), 1 h for MEGAN4 (interactive mode), 6 h for taxator-tk (∼10 CPU cores) and almost a week for CARMA3 (∼20 CPU cores). The parallelization of taxator-tk led to a linear decrease in time with the number of CPU cores for up to 15 cores, which became sublinear for 20 cores or more . To provide a more specific estimate of the throughput of taxator-tk, we aligned ∼1 Gb of cow rumen sequence data with BLAST against mRefSeq54 and assigned the data with taxator-tk on 10 CPU cores (AMD Opteron 6386 SE). We measured an average throughput of 5.9 Gb per day for the combined alignment and taxonomic assignment steps with this dataset. We also determined how our implementation scaled for increasing input sequence lengths and reference exclusion scenarios . The run-time scaled approximately linearly except when the same or very similar species were among the reference genome sequences. In general, the greater the number of similar sequences in the reference data, the longer taxator-tk’s run-time was for the alignment of longer sequence stretches with more homologs. Simultaneously, we investigated the impact of the query segmentation on taxator-tk’s run-time  and found that it reduced the total run-time by up to 30%.The run-time for the taxonomic metagenome assignment was measured as the time to find homologs and to assign taxon IDs to all sequences. We evaluated the run-times of all methods using the same set of alignments generated with either nce data . This toTaxator-tk is a taxonomic assignment software package which generates very precise taxonomic assignments with few errors for metagenome shotgun sequences. To provide a fair comparison, we invested extensive effort into ensuring that we evaluated all methods under identical conditions with the same reference sequences, test datasets and background taxonomies, using their recommended settings. We evaluated taxator-tk on 16S gene sequences, on simulated short reads, with simulated assembled contigs and with 2 kb contig fragments from a real cow rumen metagenome. For each simulated sample, we evaluated a wide range of evolutionary distances between the query and reference sequences using leave-one-taxon-out cross-validation. Taxator-tk was the most precise of all tested methods with the most realistic number of identified taxa overall. This property was very pronounced for lower taxonomic ranks from species-to-family level. However, taxator-tk assigned fewer data overall than other methods from species to family. For the small assembled SimMC dataset, it assigned fewer data, particularly in comparison to the composition-based classifier PhyloPythiaS, when 100 kb of data were provided for individual community members to train species-level models. For the real cow rumen dataset, taxator-tk was the most consistent in terms of classifying multiple pieces of one contig. Our results consistently indicate that taxator-tk’s strength is its high precision of assignments, which allows us to confidently assign a core of sample sequences and thereby to infer the taxonomic composition of the community. In comparison to assignments based on marker genes, it has the advantages that it makes assignments across all domains of life and that corresponding abundance estimates from shotgun sequences are less affected by copy number variations of individual genes. Such shotgun estimates are also unaffected by PCR primer amplification biases, unlike marker gene sequencing techniques, and do not require high-quality reference gene phylogenies for marker genes. We confirmed this by in depth analysis of six 15 Gb shotgun samples from the barley rhizosphere, where we applied taxator-tk to characterize the taxonomic composition of bacteria, archaea and eukaryotes, which correlated with results from 16S rRNA profiling and showed the most notable deviations for taxa known to be affected by primer biases or having multiple copies of the 16S rRNA gene . To target draft genome reconstructions, the data assigned to individual taxonomic bins by taxator-tk can be used as training data for complementary approaches, such as composition-based methods, or as independent information in combination with recently proposed clustering methods using the abundance of genes or contigs across multiple samples.de-novo phylogenetic inference methods, the run-time increases at least log-linearly with the number of homologs or they rely on time-consuming optimizations of parameter-rich phylogenetic models, which generates excessive computational requirements for the analysis of Gb-sized NGS samples. Our software provides an easy to use and scalable alternative to taxonomic classification of marker genes that is applicable to any nucleotide fragment. Unlike other similarity-based taxonomic classifiers for shotgun data, our algorithm handles different degrees of sequence conservation without preset or user-specified parameters such as alignment scores  and without being restricted to the analysis of a number of high-quality homologs with a minimal length. At the same time, the inferred evolutionary neighborhood is extended by the identification of an outgroup, leading to more precise taxonomic assignments, while regions without detectable taxonomic signal are instantly discarded. We post-process independent taxonomic assignments of query segments to infer an assignment for the entire query and do this using a majority vote algorithm with a few robust default parameters. This computationally lightweight step can be quickly repeated with other values for the majority and minimum support parameters, if required. In addition to the algorithmic considerations and other run-time optimizations, we implemented query sequence segmentation and program parallelization, which allow large-scale data analysis with a throughput of several Gbs per day on a standard multiprocessor system.From a methodological point of view, we have introduced a method for the fast approximation of the evolutionary neighborhood of a query sequence with a run-time that increases linearly with the number of homologs. In taxator within taxator-tk provides taxonomic information for individual query segments , which could be used to identify assembly errors or regions acquired by lateral gene transfer.The program’s scope is also not limited to the taxonomic assignment of metagenomes: It can be applied to any DNA or RNA sequence. For instance, another successful in-house application is the detection of contaminations in isolate sequencing data. Furthermore, the program


Alignment of large and diverse sequence sets is a common task in biological investigations, yet there remains considerable room for improvement in alignment quality. Multiple sequence alignment programs tend to reach maximal accuracy when aligning only a few sequences, and then diminish steadily as more sequences are added. This drop in accuracy can be partly attributed to a build-up of error and ambiguity as more sequences are aligned. Most high-throughput sequence alignment algorithms do not use contextual information under the assumption that sites are independent. This study examines the extent to which local sequence context can be exploited to improve the quality of large multiple sequence alignments.Two predictors based on local sequence context were assessed: (i) single sequence secondary structure predictions, and (ii) modulation of gap costs according to the surrounding residues. The results indicate that context-based predictors have appreciable information content that can be utilized to create more accurate alignments. Furthermore, local context becomes more informative as the number of sequences increases, enabling more accurate protein alignments of large empirical benchmarks. These discoveries became the basis for DECIPHER, a new context-aware program for sequence alignment, which outperformed other programs on large sequence sets.Predicting secondary structure based on local sequence context is an efficient means of breaking the independence assumption in alignment. Since secondary structure is more conserved than primary sequence, it can be leveraged to improve the alignment of distantly related proteins. Moreover, secondary structure predictions increase in accuracy as more sequences are used in the prediction. This enables the scalable generation of large sequence alignments that maintain high accuracy even on diverse sequence sets. The DECIPHER R package and source code are freely available for download at DECIPHER.cee.wisc.edu and from the Bioconductor repository.The online version of this article (doi:10.1186/s12859-015-0749-z) contains supplementary material, which is available to authorized users. Multiple sequence alignment (MSA) is a ubiquitous task in biology, and has a wide variety of applications including homology detection , predictIt is generally believed that the poor scalability of alignment can be attributed to the build-up of error or the increasing level of ambiguity as more-and-more sequences are aligned. Two main strategies have been proposed to combat the loss in quality as alignments grow in size. The first strategy is to use a chained guide tree, which is efficient to construct and allows reasonable accuracy to be maintained on large empirical datasets  . HoweverMSA programs are typically optimized and assessed based on their ability to recreate the alignments in benchmark datasets. In this way, benchmarks determine the objective to which alignment programs strive to attain. There is an ongoing debate over whether simulated, structural, or other types of benchmark are preferable . SimulatIn contrast, many structural benchmarks have been built from related RNA or protein tertiary structures that have been superimposed to provide an empirical alignment that is free of many of the simplifications of simulated alignments. By this definition residues in the same column of an alignment should occupy the same structural position in space. A major downside of structural benchmarks is that “gappy” regions are typically not considered in scoring because they are not superimposable in space . Some doThe focus on maximizing true homologies has been furthered by a reliance on Q-score for performance comparisons with structural benchmarks. Q-score is defined as the average pairwise fraction of reference homologies that are also found in the test alignment . Q-score does not directly penalize for aligning positions that are unaligned in the reference, also known as over-alignment . Over-alAssessment of over-alignment is one step in the ongoing effort to create more biologically meaningful alignments . Other eHowever, these gains have come at a cost because secondary structure is time consuming to accurately predict, which prevents these methods from scaling to a large number of sequences. Presently none of the alignment programs that use predicted secondary structure can align a thousand or more sequences in a reasonable amount of time . This inIn this study, I began by comparing the accuracy of structural benchmarks that would form the foundation for the rest of the study. Next, I investigated whether it was possible to efficiently integrate secondary structure predictions with negligible added time and no additional sequences other than those being aligned. To do this I relied on less accurate, but very fast, predictions made using the GOR method  for secon sequences therefore resulted in (n2 - n)/2 different pairwise alignments. The pairwise secondary structural identity of each of these alignments was calculated and used to compare benchmarks. Secondary structure identity was defined as the number of columns with matching secondary structure (8-state DSSP) normalized by the maximum number of matches possible. The large number of data points was simplified for plotting by finding the shortest contour line on the kernel density surrounding 75 % of points. The R programming language [P-values were calculated using the Wilcoxon signed rank test in R [To compare empirical benchmarks, secondary structure assignments according to DSSP  were dowlanguage  was usedest in R .Fig. 1CoFor secondary structure predictions, the GOR method was re-implemented as the DECIPHER function “PredictHEC”, and used automatically during alignment of amino acid sequences. The GOR method was trained on the dataset in Li et al. , which wth, 2015 from the website mizuguchilab.org/homstrad. The HOMSTRAD alignments were realigned using MUSTANG (v3.2.1) [www.drive5.com/bench. This collection includes OXBench [HOMSTRAD  multiple(v3.2.1) . All oth OXBench , PREFAB  OXBench , and tra OXBench  in both  OXBench . These bTo create HOMSTRAD-mod, columns of the alignment that were in agreement between the original and MUSTANG alignments were kept uppercase to define core blocks. Therefore, HOMSTAD-mod alignments are identical to those of HOMSTRAD in the regions used in scoring. Alignments with (i) less than 25 % of their length in core blocks, (ii) a total width of less than 30 sites, or (iii) having greater than 80 % average pairwise identity were removed. Benchmarks were supplemented with full-length Pfam  sequenceWhen comparing performance, input reference sets were generated by randomly selecting a predefined number of supplemental sequences from the pool of available Pfam sequences. These supplemental sequences were added to the reference sequences to reach the intended total number of sequences in each input set . After alignment, the supplemental sequences were removed and the remaining (reference) sequences were tested for alignment accuracy. Only one randomly selected set of supplemental sequences was used per alignment size, up to the maximum number of sequences available for each set. The smallest sets of 2 sequences were created by randomly selecting a pair of sequences from each reference set. All alignments were scored using qscore  with optSequence pairs in the One Gap Database  were tralog(observed probability/background probability)*3/log(2). Log-odds scores were highly correlated between the two datasets (R2 = 0.88), so the average score was used for model parameters.To prevent incorrect gap placements, sequence pairs were required to contain gap events separated by at least 20 residues, and have greater than 50 % sequence identity. To mitigate the effect of ambiguous gap placements in repetitive regions, the sequence pairs were realigned in reverse orientation and then reversed again to generate a complementary alignment. Finally, local alignments that were equivalent  were expanded into all possible permutations and weighted to split the permutations evenly. The same process was repeated with Pfam families to generate a complementary set of high quality gap placements. The final sets contained 58,509 gaps from the One Gap Database, and 46,168 from the Pfam database. Observed residue frequencies were converted into log-odds in third-bits based on the formula: DECIPHER is an R  package Clustal Omega (v1.2.0) DECIPHER (v1.14.4)MAFFT (v7.22.0) MUSCLE (v3.8.31) PASTA (v2.2.7) PROMALS The following programs were compared in this study:Default parameters were used for all programs with the exception of MUSCLE and PASTA, which required changing the maximum number of iterations. For MUSCLE, “maxiters” was change from 16 to 2 for sets of 500 or more sequences as recommended by the developers. For PASTA, the parameter “iter-limit” was changed to 1 for sets of 500 or more sequences. Attempts to use the default value of 3 proved prohibitively time consuming on larger sets. For MAFFT the “auto” option was used to automatically switch between different progressive and iterative strategies based on the number and length of input sequences. Timings for all sets were determined on a 2.2 GHz Intel Core i7 with 8 GB of RAM using a single processor. For consistent timing comparisons, PASTA and PROMALS were configured to use only one processor.Different benchmarks often result in contrasting optimal parameters  and an incompatible performance ranking of alignment programs . For thep < 1e-15) than SABmark. These findings are in agreement with a previous study [It is expected that better reference alignments will have a greater percentage of aligned residues with identical secondary structure. However, some disagreement in secondary structure is anticipated due to both intrinsic difficulties in assigning secondary structure  and chalus study  that fouus study .p < 1e-15). This result supports the use of the HOMSTRAD database as an alignment benchmark even though it was not originally intended for this purpose.All columns of the alignments were used to assess the overall accuracy of each benchmark rather than only using core blocks (homologous regions), which are typically delineated by uppercase letters. The choice to use the entire alignment was made because: (i) the definition of core blocks varies between benchmarks, (ii) some scoring procedures make use of the entire alignment , (iii) pp < 1e-15). This indicated that extending the input set in a way that is not directly dependent on the reference sequences results in the greatest diversity of supplemental sequences.Since the number of sequences with known structure is small relative to the number of available sequences, most benchmarks are supplemented with additional unaligned sequences that are not considered in scoring. PREFAB reference alignments are supplemented with additional sequences found using PSI-BLAST searches  with theIt is unclear which reference benchmark most adequately reflects a typical user’s sequences, and the wide diversity of MSA applications probably spans most of the alignment scenarios found in benchmarks. SABmark sets cover a narrow range of sequence identities, while OXBench focuses on closely related sequences that are easier to align. Due to both alignment quality and breadth of sequence identities, I chose to continue the rest of this study with slightly modified versions of the original PREFAB and HOMSTRAD datasets, called PREFAB-mod and HOMSTRAD-mod see . To suppDespite the close connection between secondary structure and sequence alignment, most popular protein alignment programs do not predict structural information. The main drawback of secondary structure prediction is that it is slow to accurately compute, which prevents it from scaling to the alignment of hundreds of sequences in a reasonable amount of time . Less acTo integrate secondary structure predictions into the dynamic programming framework for profile-profile alignment, I added a new 3 × 3 symmetric matrix representing the log-odds of aligning an H, E, or C in one sequence with another position assigned to H, E, or C in a second sequence. Coupling this matrix with the probability assigned to each of the three structural states allowed for profile-profile alignment of the secondary structures. The score obtained from aligning secondary structure profiles augmented the traditional substitution matrix based score determined from the primary sequences  are more likely to occur adjacent to certain amino acids  and in eFigure 2 of 0.69 and 0.64, respectively). However, in this case there was no apparent difference between locations within the gapped region. For this reason I chose to simply modulate the gap extension cost based on the average scores for the “gapped” residues in a position-independent manner. Altogether, this probabilistic model of opening and extending a gap adjusts the gap penalty within a range of about +/- 20 % at each position.Next, I calculated log-odds scores for the residues opposing the gap (in the ungapped sequence), and found that these positions displayed a small bias in amino acid content  surrounding gaps were only slightly more likely (< 1 third-bit) than expected by chance. However, gaps were substantially more likely to occur around runs of three or longer , as shown in Additional file Overall, employing this model of gap placement resulted in a modest improvement of 0.5 % on PREFAB-mod (p < 1e-4) and 0.3 % on HOMSTRAD-mod (p < 1e-3). The improvements in Q-score were matched by 0.2 % increases in M-score on both benchmarks. These changes in score were unexpected, as structural benchmarks do not consider most gapped regions since they often occur in parts of the structure that are difficult to superimpose , and repHaving successfully integrated context-awareness into the DECIPHER software for sequence alignment, I next compared its performance to other state-of-the-art alignment programs. First, I chose to benchmark DECIPHER against three popular programs capable of efficiently aligning thousands of sequences: Clustal Omega , MAFFT , and MUSWhen only two sequences were aligned from each benchmark, the alignment programs all gave similar results, with MAFFT showing the lowest accuracy. In the sets of 125 sequences, DECIPHER is ranked second behind MAFFT. For input sets of this size, MAFFT uses its most accurate consistency-based algorithm (L-INS-i) that is not scalable to larger sequences sets. Beyond 125 input sequences, DECIPHER clearly outperforms the other three programs . The popular strategy employed here, of extending structural benchmarks with supplemental sequences, suffers from a dilution problem as the number of supplementary sequences begins to greatly outnumber the reference sequences.It has been previously established that the vast majority of information indicating whether to align two positions is contained directly in the amino acid pairing. This has led to the assumption of positional independence that is the primary means for efficient alignment algorithms . HoweverMy own attempts to construct a substitution matrix based on amino acid triplets also showed signs of estimation inaccuracy. However, testing this matrix did reveal a small improvement in Q-score, albeit far less than that of using secondary structure predictions. The GOR (version IV) method employed here uses two matrices of parameters, one based on single residues and the other on pairs of residues, which can be accurately estimated due to their relatively small size. Furthermore, reduction to a three-letter (H/E/C) alphabet that reflects an important property of the alignment enables local sequence context to be efficiently harnessed, because the contextual information only needs to be computed once per site and can then be reused under the dynamic programming approach to alignment. In contrast, using large substitution matrices requires re-computing the covariation score at every site, which is very inefficient and is not suitable for large sequence sets .The main finding of this study is that fast secondary structure predictions can be employed in a scalable manner to counteract the drop-off in accuracy associated with aligning more sequences. This effect can be explained by the fact that structure is more conserved than sequence and therefore remains a reliable predictor even as sequences diverge greatly. Secondary structure prediction algorithms exhibit a similar increase in accuracy as more sequences are used in the prediction. For example, accuracy of the GOR algorithm increases by 6 % when multiple sequences are used for prediction . The samThere is an inherent trade-off between true and false homologies, and the results of this study advocate for the comparison of both in the development and benchmarking of alignment algorithms. While it is common to report Q-score and TC-score, these two statistics are strongly correlated. In contrast, Q-score and M-score are not linearly related, and beyond a certain optimum one must be lowered to raise the other. Analyses of alignment performance have often focused solely on quantifying true positives , which has the potential to paint an unbalanced picture of alignment performance. Similarly, the choice of alignment benchmark was carefully analyzed in this study. The results showed that not all reference sets are equally well aligned, and therefore benchmarks should be compared in addition to alignment programs . Treatin


Motivation: Alignment-based methods for sequence analysis have various limitations if large datasets are to be analysed. Therefore, alignment-free approaches have become popular in recent years. One of the best known alignment-free methods is the average common substring approach that defines a distance measure on sequences based on the average length of longest common words between them. Herein, we generalize this approach by considering longest common substrings with k mismatches. We present a greedy heuristic to approximate the length of such k-mismatch substrings, and we describe kmacs, an efficient implementation of this idea based on generalized enhanced suffix arrays.Results: To evaluate the performance of our approach, we applied it to phylogeny reconstruction using a large number of DNA and protein sequence sets. In most cases, phylogenetic trees calculated with kmacs were more accurate than trees produced with established alignment-free methods that are based on exact word matches. Especially on protein sequences, our method seems to be superior. On simulated protein families, kmacs even outperformed a classical approach to phylogeny reconstruction using multiple alignment and maximum likelihood.Availability and implementation:kmacs is implemented in C++, and the source code is freely available at http://kmacs.gobics.de/Contact:chris.leimeister@stud.uni-goettingen.deSupplementary information:Supplementary data are available at Bioinformatics online. Comparative sequence analysis traditionally relies on pairwise or multiple sequence alignment. With the huge datasets that are produced by next-generation sequencing technologies, however, today’s alignment algorithms reach their limits. Thus, with the growing number of completely or partially sequenced genomes, there is an urgent demand for faster sequence-comparison methods. Over the past two decades, a wide variety of alignment-free approaches were proposed . Althougk, also called k-mers, in the input sequences. Other methods are based on variable-length matches; they have the advantage that it is not necessary to specify a fixed word length  method  approach has been proposed by shustring length; they implemented this approach in the program rK . k = 0, we have ACSd.In this article, we generalize this distance measure by using substring matches with n is the maximal length of the sequences. As we want to compare sequences in linear time, however, we propose a heuristic to approximate these values. To do so, we first calculate for each position i in S1 the length i matching a substring of S2, as is done in ACS. Let j be the start of this matching substring in S2; the character S1 from position S2 from k + 1-th mismatch or the end of one of the two sequences is reached.For a pair of sequences, the exact values i = 4 in S1 and with k = 2 mismatches, our approach would return the following k-mismatch common substring, starting at position j = 2 in S2:In the example below, for position k-mismatch common substring, our program would first determine the longest common substring for position i = 4 in S1 that exactly matches a substring in S2. We find such a match at position j = 2 in S2 with the length matching positions).To obtain this i in S1, the corresponding position j in S2 of the longest exact match to a substring starting at i may not be unique. Consider, e.g. position i = 2 in the first sequence of the above example:It should be mentioned that, for a position AT starting at position 2 in S1 is the longest substring starting at this position and matching a substring of S2—but this substring occurs at positions 1, 5 and 10 in S2. In such a case, we calculate all k-mismatch extensions of these occurrences as described above, and we define maximal possible extension minus k.Here, the substring k mismatch maximal substring lengths from z is the average number of maximal matches to a substring in S2 starting at a position i in S1. In principle, this complexity could be achieved by using suffix trees . In a suffix-tree approach, this could be accomplished by lowest common ancestor (LCA) queries. Similar to the aforementioned approach, we would have to look up the lowest node v that is above both leafs i and j; the string depth of v is then the length of the longest exact match starting at i and j, respectively. LCA queries can be carried out for any i and j in constant time after a linear-time preprocessing step  array for S. Here, the entry LCP[i] stores the length of the LCP of the SA[i]-th suffix and its predecessor in SA, the SASA of a sequence S together with the corresponding LCP array is called, in this context, the enhanced suffix array of S. To calculate enhanced suffix arrays in linear time, we used a program described by http://algo2.iti.kit.edu/english/1828.php. The underlying algorithm is based on sais-lite by Yuta Mori, a fast implementation of induced sorting .Position 10 in Require: SA {generalized suffix array for S1 and S2 of length n}Require: LCP {corresponding longest common prefix array}Ensure:s {stores the results of  fori = 2 to n – 1 do ifthen  ifthen       end if      else        end if  end for  fori = n to 2 do ifthen  ifthen       end if      else        end if  end for s(i) can be calculated for the entire concatenated string S in linear time using Algorithm 1. Here, the first loop computes i and stores them as s[i]. Then the second loop calculates s[i] if the result is greater than the actual value of s[i]. This way, algorithm 1 applies i and stores the corresponding values s[i].All values i all positions belonging to the respective other sequence, where a match of length s(i) occurs. This can be achieved by a simple extension of Algorithm 1. Without loss of generality, we assume that the first minimum in all indices Finally, for our heuristic we need to find for an index all such p that belong to the other sequence, the positions SA[p] are occurrences of longest substrings matching a substring starting at i. In our example, we find one further position p = 3, so SAFor S1 and S2, respectively. Using the enhanced suffix array of a sequence S, the length of the longest substring starting at positions SA[i] and SA[j] (with SASA[j]) is given as the minimum over the values LCP[p], LCA queries to obtain this value known as range minimum queries (RMQ). A RMQ returns the index of an array A that stores the smallest element between two specified indices l and r, denoted as Next, the second step in our approach involves finding the length of the longest common substring starting at pre-defined positions in RMQ in constant time, after a linear preprocessing step, e.g. i and j can be calculated as enhanced suffix arrays. In our implementation, however, we extend the substrings by matching single characters because in our test runs this ‘naive’ approach was faster than the RMQ implementation that we tested. Nevertheless, our downloadable program has an option for using the RMQ algorithm so the user can compare these two approaches.Several algorithms are available that can solve kmacs and to compare it with other methods of sequence comparison, we applied these methods phylogeny reconstruction. We used a large number of DNA and protein sequence sets for which reliable phylogenetic trees are available, and we measured how similar the constructed trees are to the respective reference trees. The following sequence sets were used in our study:To evaluate kb. A benchmark tree that has been constructed based on a multiple alignment.For eukaryotic DNA comparison, we used a set of 27 primate mitochondrial genomes that were previously used by Roseobacter genomes, which were previously analysed by mb.As prokaryotic genomes, we used a set of 32 BAliBASE (v3.0) database    as well FFP, rK and CVTree return pairwise distances between the input sequences. For ACS, we calculated distances as defined in (3), and for spaced words and the k-mer approach we used the Jensen–Shannon divergence  distances to the reference trees. For spaced words, we used the same value for k, even though better results might be possible with different values. Accordingly, on every group of benchmark data, we tested FFP, CVTree and UA with different parameter values and used those which produced the best results on this group.vergence , appliedNeighbor joining  with the alignment-based reference tree from kmacs almost coincides with the topology of the reference tree; the RF distance between these trees is 2.Roseobacter genomes, the best methods were kmacs with k = 4 and 6, FFP and CVTree as shown in Spaced words and the generic k-mer approach performed slightly worse. None of the tested methods was able to exactly reconstruct the topology of the reference tree. UA is missing in this comparison, as this program is too slow to be run on full bacterial genomes in reasonable time. For our simulated DNA sequence sets, the results were similar as for the primate mitochondrial genomes; see spaced words was the best alignment-free method, followed by kmacs. This time kmacs outperformed the established alignment-free approaches for all values of k that we tested. On our simulated DNA sequences, we could also run a classical approach to phylogeny reconstruction using Clustal W and Maximum Likelihood. Not surprisingly, this slow and accurate method performed better than all alignment-free approaches.On the BAliBASE protein sequences. Spaced words and kmacs again produced better results than the existing alignment-free methods that we evaluated. This time, there was a large range of values for k where kmacs performed similar or even slightly better than spaced words, and both methods outperformed the other alignment-free methods that we tested. As with the previous dataset, the classical approach based on multiple sequence alignment performed best; this time the difference between alignment-based and alignment-free methods was larger. This may be because of the fact that multiple-alignment programs are often tuned to perform well on BAliBASE, the main database to evaluate multiple-alignment methods.spaced words and kmacs outperformed other alignment-free approaches and, as on BAliBASE, kmacs was slightly better than spaced words if k was sufficiently large. Surprisingly, on these benchmark sequences spaced words and kmacs even outperformed Clustal W and Maximum Likelihood, although not dramatically.Finally, the results on our simulated protein sequences are shown in indirectly, by applying them to phylogeny reconstruction and comparing the resulting trees with trusted reference trees using the RF metric. This is a common procedure to evaluate alignment-free methods. RF distances to reference trees are only a rough measure of accuracy, though, as they are based on tree topologies alone and do not take branch lengths into account. Furthermore, the constructed trees depend not only on the underlying methods for sequence comparison but also on the methods used for tree reconstruction. A more direct and accurate way of comparing alignment-free methods is to directly compare the distance values that they calculate. This can be done, for example, by plotting the distances produced for simulated sequences against their real evolutionary distances  as estimated with our heuristic and (b) calculated with a slow and exact algorithm. The figure shows that our heuristic is clearly suboptimal. But the goal of our project was not so much to precisely estimate the maximal k-mismatch substring lengths, but rather to define a distance measure on sequences that can be efficiently calculated and that can be used to obtain biologically meaningful results. Therefore, we think that the discrepancies between the optimal substring lengths and the values estimated by our heuristic are acceptable. k-mismatch common substring lengths might improve the sensitivity of kmacs on divergent sequence sets because the curves for the exact solutions converge at higher substitution frequencies. In fact, on the mitochondrial genomes that we used as benchmark data, an exact algorithm led to better phylogenetic trees than our greedy heuristic . Therefore, it may be worthwhile to develop heuristics that approximate the maximal k-mismatch substring lengths more accurately.Finally, we wanted to know how accurately our greedy heuristic approximates the exact word matches. In this article, we presented a novel alignment-free algorithm that takes mismatches into account. This is similar in spirit to the spaced-words approach that we previously proposed . It seems therefore best to apply kmacs to distantly related sequence sets, while methods such as rK and ACS may be preferred on evolutionarily more closely related sequences.In our program evaluation, we used DNA sequence sets with large evolutionary distances. On these sequences, our new alignment-free methods performed better than established methods that rely on exact word matches. Algorithms using exact matches, on the other hand, seem to work better on smaller evolutionary distances. exact matches between local homologies can usually be extended until the first substitution is reached. The average length of longest common substrings and of shortest unique substrings, respectively, can therefore be used to estimate substitution probabilities (kmacs as long as k is small enough. In this case, all k mismatches are likely to be used up in a k-mismatch common substring extension before the first indel occurs. Thus, the average length of the longest k-mismatch common substrings depends on the frequency of mismatches and could be used to estimate substitution probabilities, just as in rK.In biological sequences, substitutions are more frequent than insertions and deletions. Consequently, k is sufficiently large, substring matches between local homologies are essentially extended until the first indel occurs. From this point on, the mismatch frequency is high and the remaining mismatches will be used up quickly. So in this situation, the average k-mismatch substring length depends on the frequency of indels rather than on the frequency of substitutions. This may explain why ACS and rK work well on closely related sequences, while kmacs is superior on distantly related sequences where the frequency of indels may be a better measure for evolutionary distances than the frequency of mismatches.In contrast, if kmacs and the quality of the phylogenetic trees that we could produce with it, our approach should be useful not only for fast phylogeny reconstruction, but also for other tasks in comparative sequence analysis.In our study, we used alignment-free methods to reconstruct phylogenetic trees and evaluated the quality of these trees. But phylogeny reconstruction is only one important application of sequence comparison. Clustering, classification and remote-homology detection are other fundamental challenges in DNA and protein sequence analysis. With the rapidly growing size of sequence databases, alignment-free methods have become indispensable for these tasks . Given t


Ecoregionalization of the ocean is a necessary step for spatial management of marine resources. Previous ecoregionalization efforts were based either on the distribution of species or on the distribution of physical and biogeochemical properties. These approaches ignore the dispersal of species by oceanic circulation that can connect regions and isolates others. This dispersal effect can be quantified through connectivity that is the probability, or time of transport between distinct regions. Here a new regionalization method based on a connectivity approach is described and applied to the Mediterranean Sea. This method is based on an ensemble of Lagrangian particle numerical simulations using ocean model outputs at 1/12° resolution. The domain is divided into square subregions of 50 km size. Then particle trajectories are used to quantify the oceanographic distance between each subregions, here defined as the mean connection time. Finally the oceanographic distance matrix is used as a basis for a hierarchical clustering. 22 regions are retained and discussed together with a quantification of the stability of boundaries between regions. Identified regions are generally consistent with the general circulation with boundaries located along current jets or surrounding gyres patterns. Regions are discussed in the light of existing ecoregionalizations and available knowledge on plankton distributions. This objective method complements static regionalization approaches based on the environmental niche concept and can be applied to any oceanic region at any scale. The ecoregionalization of the ocean is useful for scientific research, conservation and management of the marine environment and marine resources. For instance, ecoregionalization is needed to extrapolate punctual or transect data to broader areas and to target specific regions for interdisciplinary research  were taken from the configuration PSY2V3 of the operational system MERCATOR OCEAN st 2007 to December 25st 2009 in order to fully sample the variability of the circulation. This represents a total of 8,309,304 particles for surface depth, respectively 7,701,480 for depths 50 and 100 m. The choice of 10 km and 3 days is a compromise between matching the horizontal resolution of the model, taking into account mesoscale processes and keeping an affordable computing time of resulting trajectories. We thus obtained three ensembles of trajectories, one per depth.The trajectories followed by numerical particles were calculated offline with the Lagrangian diagnostic tool ARIANE In order to quantify the connections over the model domain, the domain was divided into grid cells of 50 km×50 km on a regular square grid, the connectivity grid, with a total of 1095 cells covering only regions with depths greater than 100 m. The 50 km resolution is sufficient to keep a reasonably realistic coastline while being suitable with the seeding density chosen. Thus each connectivity grid cell contains 5*5 = 25 particles for each initial seeding date, except grid cells including land that contains less particles.i to grid cell j, MCT was computed asTo quantify the connectivity between each grid cell, we used the Mean Connection Time, hereinafter MCT. Defining T as the transit time from grid cell M being the number of particle transitioning from i to j. Note that for each trajectory, all intermediate transitions were used to compute the MCT. The sensitivity of MCT to the number of particles was tested. The suite of MCT matrices converged when the number of particles was greater than 6,000,000, therefore we considered that 8,309,304 particles and respectively 7,701,480 particles for depths 50 and 100 m were sufficient to obtain a robust MCT matrix. Moreover, to keep MCT robust, it was computed only when M was greater or equal to 50. Four MCT matrices of size 1095×1095 were computed: one MCT matrix from each ensemble of trajectories  and also one MCT matrix using the three ensembles together (MCT3depths).Not all grid cells of the domain were connected within one year, especially remote cells . Thus the resulting MCT matrices had gaps (from 37% to 56%). These gaps are a problem for the steps of computing the oceanographic distance and applying hierarchical clustering on it. Therefore a gap filling procedure was introduced as follows see :M.○ For each unconnected pair of grid cells i-j, we looked for grid cells k so that i-k and k-j pairs are connected. There must be at least 50 grid cells k as for ○ Then we computed MCT for pair i-j as the sum of the MCT and MCT, averaged on all existing cells k, and filled the MCT value in the matrix.After 3 iterations of this procedure, each MCT matrix was filled. The resulting MCT values ranged from 10 days to 3000 days. This gap filling procedure avoided the very long integration time (>8 years) needed if we were to fill the whole MCT matrices from original trajectories alone.i to j is not equal to the time to go from j to i. Then the oceanographic distance (OD) was defined after i and j (travel from i to j and return travel from j to i). We chose the minimum value as it corresponds to the fastest route of transport which is also the shortest in length.This led to four full MCT matrices, which are asymmetric since the time to go from 0, OD50, OD100, OD3depths) where all diagonal terms (autoconnection time) were set to zero.This gave four symmetric matrices,  is not computed with a distance metric applied to a given dataset. Thus, usual criteria based on dataset variance within clusters cannot be used e.g.  because When the number of clusters increases, the within-cluster MCT diminishes, as well as the size of each region  and 4. TOn the basis of our interpretation of regions with respect to circulation, we retained 22 clusters . The bouThen considering the 22 regions, the Western basin is separated into eight regions; regions A and B in the Northern part of the basin, G, F and E in the South and C, D that contains the Tyrrhenian sub-basin, region H at the center. In the Eastern basin, the Adriatic Sea is one region I. The Ionian Sea is separated into regions J, V, T at the center and K to the east, with U and S along the coasts of Libya and Tunisia. The Aegean Sea is divided into two regions, M in the East, L in the West. The Levantine basin has four regions: two coastal regions N and O, one southern region P and one center region Q. Considering only stable boundaries, the Western basin only has 5 regions. The Eastern basin has few continuous boundaries, only 4 regions are delimited .The boundary stability shows that the majority but not all boundaries are robust to changes in linkages and depths. Often, linkages or depth changes can produce minor shifts in boundary position, hence reducing boundary stability as defined here. When a boundary is not stable, it means that either the circulation is variable, either it is located in a region where the distance (OD) among grid cells is small thus the boundary position varies according to the overall content of each cluster. Thus the boundary map must be analyzed jointly with the boundary stability to assess our regionalization.First the meaning of the regions obtained needs to be explained. One region contains grid cells that are connected at shorter time scale with each other than they are to the grid cells of the other regions. In the following, the relationship between the clusters boundaries, their stability and the circulation is examined in detail in comparison with the model average velocity fields  and liteThe hierarchy of cluster boundary is in good agreement with the surface general circulation scheme proposed by Millot et al. In the Western basin, boundary #16 is associated to the path of the Northern Current In the Eastern basin, the Ionian Sea has two Southern regions U and S. Boundary #10 follows the Sicilian current of AW and region U contains the area of accumulation of eddies of the Ionian Sea Some regions are virgin of any boundaries , like thThus this regionalization reveals known circulation patterns and summarizes them in a way that complements the simple average velocity field analysis. It can be used to quantitatively compare the circulation patterns from contrasted periods or from different models.The identification of regions close to each other, not geographically but in terms of oceanographic connections, should help understanding the spatial distribution of properties that are passively transported by currents, such as conservative physical properties, or planktonic organisms living in the surface layer (epipelagic).First, boundaries emerging from circulation alone often match major discontinuities in variables describing the environment. For instance a strong latitudinal salinity gradient exists near the Balearic Islands, close to our boundary #6. However, our boundary #6 coincides with the Balearic Current but not to the Balearic salinity front, located more to the South Within our regions, planktonic organisms are connected at shorter time scales than between regions. Thus hydrodynamical boundaries can become faunistic boundaries as suggested by Gaylord and Gaines Ecoregions drawn qualitatively from expert knowledge of species assemblages  also dis. Moreover, we deal with particles in the 0–100 m layer, which only properly represent epipelagic zooplankters dispersal.However, for living organisms such as zooplankton, circulation alone is not sufficient to explain the distribution of a given species as it is adapted to its environment, in particular to a temperature range, e.g. Palunirus elephasLipophrys trigloides [48) the adequate cluster number is ca 30. The lower bound time scale we can address with the present regionalization (∼10 days) is set by the spatial resolution of our connectivity grid. Shorter time scales could be achieved with a finer connectivity grid.To use this regionalization, the question of the number of clusters to retain will arise. With our approach, no existing criterion is available to define the optimal number. However the number of clusters can be chosen based on the time scale we are interested in, as regions isolated at a given time scale become connected at a larger time scale. Therefore the time scale of interest defines the appropriate cut-off distance and the resulting cluster number and sizes . For insFew existing studies can be compared to our regionalization because the approach is original. In the Mediterranean Sea, Andrello et al. This new regionalization method quantifies the dispersal range of organisms, This dispersal dimension was shown to explain species distribution e.g.  and is tAlso, our regions illustrate why plankton organisms may be encountered outside their optimum range .Finally this regionalization is useful as a framework to interpret the genetic differentiation of a given species sampled throughout the Mediterranean e.g. . FurtherThe regionalization proposed here will eventually be compared to an ongoing biogeochemistry-based regionalization Concerning the methods, several points can be made. With a similar approach but shorter simulations, we can explore the seasonal variability of clusters boundaries that may be significant Appendix S1Method to fill the gap of the MCT matrix.(DOC)Click here for additional data file.Appendix S2Method for choosing the optimal cut-off distance.(DOC)Click here for additional data file.


Motivation: Within bioinformatics, the textual alignment of amino acid sequences has long dominated the determination of similarity between proteins, with all that implies for shared structure, function and evolutionary descent. Despite the relative success of modern-day sequence alignment algorithms, so-called alignment-free approaches offer a complementary means of determining and expressing similarity, with potential benefits in certain key applications, such as regression analysis of protein structure-function studies, where alignment-base similarity has performed poorly.Results: Here, we offer a fresh, statistical physics-based perspective focusing on the question of alignment-free comparison, in the process adapting results from ‘first passage probability distribution’ to summarize statistics of ensemble averaged amino acid propensity values. In this article, we introduce and elaborate this approach.Contact: d.r.flower@aston.ac.uk Determining the similarity between macromolecules is central to bioinformatics. While comparison of 3-dimensional macromolecular structures remains an active area, most work focuses on macromolecular sequences. From sequence similarity devolves much of our understanding of evolutionary homology and probable structural and functional relatedness, allowing sequences to be grouped in a meaningful way. It is the basis of inherited or inferred functional annotation, allowing us to deduce the broad function of proteins in newly sequenced genomes.Similarity is determined, almost exclusively, through the alignment of sequences as text. Textual sequence similarity is taken as a surrogate for common ancestry and, by extension, functional and structural similarity. Most approaches to protein sequence similarity use models of sequence evolution and compare amino-acid strings, searching for linear conservation of sequence.Sequence alignment, where equivalent or near-equivalent symbols are brought into register, has been investigated intensely for many decades , and thuThe log-odds matrices  derived While probabilistic extensions to sequence alignment, such as profiles and Hidden Markov Models (HMMs), can capture position-specific variation in multiple alignments, and typically demonstrate significantly augmented sensitivity, all alignment methods remain prone to similar limitations.Compared to sequence alignment, alternative approaches, grouped together as alignment-free techniques, and first proposed by inter alia, three , and wavelet analysis, each with their own characteristics. Smoothing seeks to reduce the random component of the initial value spectrum generated by windowing, with the high-frequency regions removed, leaving only dominant low-frequency modes.The value of individual propensity plots is limited. It works well, say, for predicting transmembrane regions within proteins, where peaks in the plot can correlate well with regions highly enriched in hydrophobic residues. Hitherto, it has proved difficult to interpret such plots other than by a peak-spotting. Several decades ago, most predictors were based on identifying maximally valued regions of sequences; essentially looking for peaks, or troughs, in some form of a propensity plot. Epitopes, such as immunological T- or B-cell epitopes ; loops aIn what follows, we take a decisive step away from such analyses, using techniques drawn from statistical physics. Specifically, we use techniques from the rich literature of first passage probability distribution, sometimes referred to as the ‘persistence’ problem ; applyinIn this article, we develop and analyze propensity data by modeling sequences as a time-series, estimating the scaling regime of a generalized probability density function (PDF) of a variable derived from the original propensity data structure. Our protocol enables us to abstract key features from propensity plots while remaining free of any text-based alignment scheme. We then apply this alignment-independent approach to the analysis of protein sequences, evaluating it as a potential means of automatically characterizing and clustering large numbers of sequences.X(t): in our case, ensemble-averaged propensity scale data. The basic question asked is: what is the probability p(t) that the field X(t) has not changed sign up to time t, starting from an initial configuration X(t0)? An equivalent question would be: what is the probability p that the field X(t) changes sign N times between t1 and t2 for X(t) > <X>? In our case, <X> = X0 or the mean value of the time series data X(t). Depending on the nature of the ‘random walk’—in this case a protein sequence—this probability is given by p ∼ (t1/t2)-m; where the exponent ‘m’ assumes different values depending on whether this probability is calculated using data above or below a certain threshold. The threshold is typically the mean value (X0) of the stochastic data. Thus multiple ‘order parameters’ may emerge from the same description depending on the chosen value of ‘m’.The idea underlying persistence analysis is simple. It relies on the nature of the PDF of stochastic time series data X0 of the data points X(t) where ‘t’ is our metaphor for the location of the data point in the time series-like sequence. What this ‘first passage statistics’ captures is the distribution of the ‘return times’ of the fluctuating ‘time series’ across the given threshold X(t) = X0.Our approach utilizes ‘extremal value statistics’ to analyze sequence structure. This builds on a tacit first approximation that sequences—as represented by propensity plots—can be seen as being predominantly stochastic in structure, at least when viewed synoptically. At this stage, the focus is on a specific manifestation, that of the ‘first passage probability’ distribution around the mean value X(t) = X0 to one above this line and then back again to X(t) < X0. As an example, t+(1) defines the first ‘plus’-type return time where ‘plus’ refers to the regime X(t) > X0 such that the time series starts from a point below this line (X(t) < X0) and after crossing this line returns back to the regime X(t) < X0. Similarly, t+(2) refers to the second such ‘plus-type’ return time; and so on.t-(1) refers instead to the first ‘minus-type’ return time where the time series count starts at X(t) > X0, then crosses the threshold line in to the X(t) < X0 regime and finally returns to X(t) > X0.t+ and t- transitions. Such a probability distribution is achieved by calculating histograms of ‘return times’ above (t+) and below (t-) the line X(t) = X0. The histograms are then normalized to obtain the desired PDFs. We assume that sequences are, to a first approximation, ‘inherently random’. As long as we are within the stochastic realm, all prior statistical results, including that for long range correlated stochastic data, suggest the existence of universal power law exponents defining the first passage probability distribution statistics related to t+ and t- transitions. In our description, we will use m+ to identify the exponent for X(t) > X0 and m_ for the exponent characterizing X(t) < X0. We now focus on this two dimensional  plane where n = +,−.Our interest is in the probability distributions of the Here, we characterize protein sequences using an alignment-free approach based on techniques of time series analysis commonly-used in statistical physics and elsewhere .m amino acids that are to be averaged upstream of the target residue, and will have n amino acids downstream, plus the value for the residue itself; all residues within the window will have independent and arbitrary coefficients (w) that will selectively weight individual positions within the travelling window. Averaging can be undertaken several times: values averaged over any particular iteration being the values generated by averaging in the previous iteration.In general, a window could adopt any arbitrary shape; thus we may assume a generalized window will have this form:N column vectors, where each column vector represented a sequence. The number of entries in each of these column vectors was identical. The block structure is a matrix M (N × 544) where each element of this matrix is a column vector.We extracted 544 useable propensity values from AAIndex . This is akin to statistical ‘ensemble averaging’, including any cogent non-ergodicity of the ensemble. The order parameter m thus represents each protein sequence as a single number. The resulting m-values were then used to cluster the sequences using the scheme detailed below. The schematic algorithm for calculating the values m+ and m- value for each of N sequences is shown below:m+, m-) calculated for each of the 544 column vectors of row i of the M-matrix for all 544 vectors in row i of the M-matrixEnsemble average  taken of Steps 1–2 repeated For each sequence drawn from Pfam  of the t+ (or t_) PDFs. The ultimate objective of this analysis was a clear clustering of the protein sequences such that similar proteins fall into separate clusters that are defined by separate combinations of  values (coordinate location in the nm – i space) where n = +, − and i = position of the protein sequence along the x-axis. Clustering follows the logic below:m+, m-) versus relevant sequence number to generate the phase diagramPlot  versus sequence number i in the phase diagramGrouping of *di between every two points representing each sequence.Starting from the phase diagram defined in 2, estimate df versus i using a first adjustable threshold (mean of the separation distances in the phase diagram plot) that separates out the large f's form the small f's. Use dendogram based MATLAB clustering protocolPlot Since the dendogram generates more clusters than the system actually has, use a second threshold (standard deviation of the data points in the phase diagram plot) and repeat step 2. The clustering accuracy will be shown by the level of gradient equality see Our ‘order parameter’ is thus the mean of the previously defined Clustering data are then plotted sequentially to evaluate the size of successive clusters. The PDFs of cluster lengths exhibit power-law scaling. More importantly, the scaling exponents can be used to compare the clustering accuracy of the Matlab-inbuilt architecture. Comparing this to known clusters provides the probabilistic values of the two external parameters determining the eventual cluster quality.We extracted one arbitrary but representative sequence family from each superfamily in the Pfam database  with an (super-diffusive) exponent p ∼ 1. The two (threshold) parameters are tuned to predict the optimal fit straight line from our algorithm parallel to that of the ‘known’ data from panel 1. To the best of our knowledge, no extant procedure has such high accuracy with only two tunable parameters.The distribution follows a power-law profile , data to be modeled , and an appropriate data representation. Data modeling methods have reached sufficient maturity, and data quality is constrained by its availability on a case-by-case basis, so increasingly it is the choice of data representation that is the crucial arbiter of success. This is especially true for protein regression, which lags far behind equivalent work for small molecules. Alignment-independent similarity measures, such as our representation, offer an interesting and seemingly productive avenue for achieving progress in this endeavor.To go beyond sequence motifs and profiles, HMMs and like methodology—and thus identify common function, structure, and evolution—new, distinct, yet complimentary, methods must be devised: alignment-free methods that can work with textual alignment to identity similarity manifest as shared structure and function.In this article, we have used results from statistical physics to address alignment-free comparison, adapting results from ‘first passage probability distribution’ to derive a single summary value able to differentiate sequence groups with high accuracy at several levels of granularity. This approach is potential highly robust being largely independent of fluctuation in the tunable parameters. We anticipate that this approach will ultimately take its place alongside textual alignment as a strongly complimentary method for sequence analysis, with many advantages compared to conventional techniques.


However, from the same set of sequences different MSAs are often inferred, depending on the methodologies used and the assumed parameters. Much effort has recently been devoted to improving the ability to identify unreliable alignment regions. Detecting such unreliable regions was previously shown to be important for downstream analyses relying on MSAs, such as the detection of positive selection. Here we developed GUIDANCE2, a new integrative methodology that accounts for: (i) uncertainty in the process of indel formation, (ii) uncertainty in the assumed guide tree and (iii) co-optimal solutions in the pairwise alignments, used as building blocks in progressive alignment algorithms. We compared GUIDANCE2 with seven methodologies to detect unreliable MSA regions using extensive simulations and empirical benchmarks. We show that GUIDANCE2 outperforms all previously developed methodologies. Furthermore, GUIDANCE2 also provides a set of alternative MSAs which can be useful for downstream analyses. The novel algorithm is implemented as a web-server, available at:  Multiple sequence alignment (MSA) is a key component in almost every comparative analysis of biological sequences (DNA or proteins). Moreover, MSA reconstruction is often the first step in bioinformatic pipelines, where MSA is later used for further analyses. Over the years, many algorithms and approaches aiming at constructing such alignments have been developed, showing a steady improvement in the accuracy of the resulting MSA –10. HoweAlignment inference is a complicated statistical estimation problem, in which alignment uncertainty originates from both the stochastic nature of the evolutionary process and computational limitations of current evolutionary models and alignment methodologies. The substantial uncertainty when inferring optimal MSAs is manifested by the large differences in the resulting alignments among existing alignment algorithms . Thus, iSeveral methods aimed at estimating unreliable alignment regions were previously developed –31. AmonAnother class of alignment reliability methods is based on a consistency principle: alignment regions that are shared among a large number of alternative MSAs built from the same sequence data are considered to be more reliable. Such consistency-based approaches differ in the way these alternative MSAs are generated. The heads or tails (HoT) methodology ,21 generMany of the above described methods were only recently developed and most were shown to outperform Gblocks, the classic and most commonly used alignment reliability methodology. In this study, we aimed to systematically compare seven of the more recent algorithms to detect unreliable regions, GUIDANCE , HoT 2121, ALISCThe performance of various MSA reliability assessment methodologies was tested on empirical benchmarks as well as on simulated data, which differ in various parameters such as number and length of sequences and average sequence identity (Supplementary Table S1). Two empirical benchmark data sets were used: BAliBASE 3  and HOMSSequences were simulated using INDELible . In ordeWe used two additional simulated data sets previously generated to evaluate MSA reliability methods. Specifically, we simulated sequences using ROSE , accordiMAFFT version 7.123b , PRANK vEight reliability evaluation methods were tested. ALISCORE version 2.0, TCS and ZORRO were applied with default parameters. Noisy release 1.5.12 was run with option ‘–seqtype P’ to indicate protein sequences. trimAl version 1.2 was run with options ‘-sgc’ and ‘-scc’ to print gap percentage count for columns in the input alignment and conservation values for columns in the input alignment, respectively. trimAl, by default, outputs filtered MSA. However, here all MSA reliability algorithms were compared by testing the agreement between the reliability score of each position and whether this position is ‘true’ or not. trimAl decides whether or not to filter a position based on a gap percentage count or based on a conservation value. When we compared trimAl to other methods we used the conservation values rather than the gap percentage count as the trimAl's reliability score for each position, as it resulted in more accurate inference. GUIDANCE version 1.5; HoT version 1.6 and GUIDANCE2 were run with default parameters. Out of the possible scores calculated by TCS, GUIDANCE, HoT and GUIDANCE2, the performance was evaluated using the column score (CS).vice versa. It is therefore common to quantify a predictor quality by considering the true positive rate and false positive rate over all thresholds. This information is given in the receiver operating characteristic (ROC) curves . Notably, the matching between columns of any two alignments (e.g. true and inferred MSAs) was computed by representing each MSA by a C matrix (the supplementary information of ). Each r) curves . ROC and) curves . The are) curves .To further improve the GUIDANCE methodology, we carefully inspected erroneous columns in the MSA that were ranked as reliable when using GUIDANCE. The characterization revealed that many errors occurred in regions containing long stretches of gaps in a substantial number of sequences in the MSA. This observation suggested that considering alternative alignments generated by varying the gap opening penalty in the alignment program can be beneficial for estimating the MSA reliability. We hypothesized that it will be beneficial to combine the three following sources of uncertainty: co-optimal solutions (as used to generate perturbed MSAs in HoT); guide tree instability (GUIDANCE); and opening gap penalty (as described above) and thus we incorporated these three components in GUIDANCE2. Specifically, given a set of sequences, a reference MSA is built using default or user-defined parameters. Next, a set of alternative alignments is created by inducing perturbations using all the above three components. Uncertainty in the guide tree is generated by computing 100 bootstrap trees  using NJIt was previously shown that alignment programs often err . Here, wWe systematically compared the performance of various MSA reliability methods: GUIDANCE2, GUIDANCE, HoT, ALISCORE, trimAl, Noisy, TCS and ZORRO both on empirical and simulated protein sequences. As can be seen in Figure It seems that for ZORRO, TCS and ALISCORE the performance drops for the OrthoMaM simulated data set, when measured in terms of AUC-ROC. Carefully observing wrong positions that were scored as reliable showed that such positions are usually ‘gappy’ positions. We thus tested the performance of ZORRO, the best method among those showing poor results on OrthoMaM simulations, and of GUIDANCE2 on positions with various gap percentages. Our results clearly show that on positions with extensive gappiness the performance of ZORRO is very poor Figure . This reAs an improved combined method, GUIDANCE2 outperforms all its components when considered separately on MAFFT alignments Figure : relayinGUIDANCE2 (similar to GUIDANCE and HoT) is not merely a filtering methodology, but rather, it allows obtaining a set of alternative MSAs for the analyzed sequences. Such alternative MSAs can be utilized to account for MSA uncertainty in downstream analyses. We first tested whether alternative MSAs generated by GUIDANCE2 have comparable average CS, as compared to the base MSA produced by the alignment program. For each alternative MSA we computed the difference in average CS with the base MSA, denoted as ΔCS. The distributions of these values over all BAliBASE MSAs and OrthoMaM simulated data are shown in Supplementary Figure S3. Notably, 40% of alternative MSAs in BAliBASE and 25% of alternative MSAs in the simulated data set showed higher average CS than the corresponding base MSAs. This result suggests that GUIDANCE2 produces biologically reasonable alternative MSAs.http://guidance.tau.ac.il. GUIDANCE is a popular user friendly web-server, in which the user can reconstruct an MSA for proteins, nucleic acids or coding sequences  relaying on a single best alignment, (ii) relaying on a single best alignment after filtering, (iii) accounting for reliability by column weighting and (iv) weighting over multiple alternative alignments. Specifically, GUIDANCE2 provides all these options, including a set of alternative MSAs which can be utilized to account for MSA uncertainty in downstream analyses. Which is best depends on the specific application and even for a specific application, debates exist, e.g. whether or not to filter alignment columns prior to tree search ,51,52.The MSAs reliability methodologies were tested on both empirical and simulated data. There are conceptual differences between simulated and structural based data sets . While tSupplementary Data are available at NAR Online.