C++ software, RNAmountAlign, for RNA sequence/structure alignment that runs in O(n3) time and O(n2) space for two sequences of length n; moreover, our software returns a p-value  based on Karlin-Altschul statistics for local alignment, as well as parameter fitting for local and global alignment. Using incremental mountain height, a representation of structural information computable in cubic time, RNAmountAlign implements quadratic time pairwise local, global and global/semiglobal (query search) alignment using a weighted combination of sequence and structural similarity. RNAmountAlign is capable of performing progressive multiple alignment as well. Benchmarking of RNAmountAlign against LocARNA, LARA, FOLDALIGN, DYNALIGN, STRAL, MXSCARNA, and MUSCLE shows that RNAmountAlign has reasonably good accuracy and faster run time supporting all alignment types. Additionally, our extension of RNAmountAlign, called RNAmountAlignScan, which scans a target genome sequence to find hits having high sequence and structural similarity to a given query sequence, outperforms RSEARCH and sequence-only query scans and runs faster than FOLDALIGN query scan.Alignment of structural RNAs is an important problem with a wide range of applications. Since function is often determined by molecular structure, RNA alignment programs should take into account both sequence and base-pairing information for structural homology identification. This paper describes  In contrast, these distance measures have low Pearson correlation when computing distances between structures taken from Boltzmann ensembles of different RNA sequences of the same length—so-called inter-ensemble correlation. For instance, the intra-ensemble correlation between base pair distance (BP) and mountain distance (MD) is 0.822, while the corresponding inter-ensemble correlation drops to 0.210. Intra-ensemble correlation between string edit distance (SE) and the computationally more expensive tree edit distance (TE) is 0.975, while the corresponding intra-ensemble correlation drops to 0.590—see A number of different metrics exist for comparison of RNA secondary structures, including base pair distance (BP), string edit distance (SE) , mountaiRNAmountAlign, uses this distance measure, since the Boltzmann ensemble of all secondary structures of a given RNA of length n can represented as a length n vector of real numbers, thus allowing an adaptation of fast sequence alignment methods. Depending on the command-line flag given, our software, RNAmountAlign can perform pairwise alignment,  by parameter fitting for the normal (ND), extreme value (EVD) and gamma (GD) distributions.Due to poor inter-ensemble correlation of RNA secondary structure metrics, and the fact that most secondary structure pairwise alignment algorithms depend essentially on some form of base pair distance, string edit distance, or free energy of common secondary structure, we have developed the first RNA sequence/structure pairwise alignment algorithm that is based on  mountain distance. Our software, h global , Smith-Wh global  or semigh global  alignmen costs gi [resp. ge]. For query a = a1, …, na and target b = b1, …, mb, define (n + 1) × (m + 1) matrices M, P, Q as follows: Mi,0 = g(i) for all 1 ≤ i ≤ n, Mj0, = 0 for all 1 ≤ j ≤ m, while for positive i, j we have Mi,j = max , Pi,j, Qi,j), where sim is a similarity measure formally defined later in i ≤ n, 1 ≤ j ≤ m, let Pj0, = 0 and Pi,j = max, and define Qi,0 = 0 and Qi,j = max. Determine the maximum semiglobal alignment score in row n, then perform backtracking to obtain an optimal semiglobal alignment.Quadratic time alignment using affine gap cost is implemented in h method  with thefound in . Let g, where 1 ≤ i < j ≤ n, such that:i, j) ∈ s then ai, aj form either a Watson-Crick  or wobble  base pair,if  ∈ s then j − i > θ = 3 (a steric constraint requiring that there be at least θ = 3 unpaired bases between any two positions that are paired),if  ∈ s then for all i′ ≠ i and j′ ≠ j,  ∉ s and  ∉ s (nonexistence of base triples),if  ∈ s and  ∈ s, then it is not the case that i < k < j < ℓ (nonexistence of pseudoknots).if , obtained from the Rfam database , is give> AJ550901.1/282-33412345678901234567890123456789012345678901234567890123GAAGAGUCGCGCUAAGCGCACUGAUGAGUCUUUGAGAUAAGACGAAACUCUUC.((((((.((((…))))……‥((((……‥))))…)))))).Positions 1 and 53 (for instance) are unpaired, as indicated by a dot, while positions 2 and 52 are paired and form the outermost base pair , positions 12, 16 are paired and base pair  constitutes one of the two apical (hairpin) loops, while the other apical (hairpin) loop is closed by the base pair , etc.a = a1, …, na, the base pairing probabilities Z is the partition function, a, E(s) is the free energy of secondary structure s using energy parameters from , defined by ensemble structural alignment distance for A by summing d1 over all aligned positions i, j for which neither character is a gap symbol, then adding positive weight w(k) for all size k gaps. By Eqs , is obtained by multiplying d1 and w(k) by −1:RNAmountAlign combines both sequence and ensemble structural similarity. Indeed, −|ma(i) − mb(j)| ∈  with maximum value of 0 while RIBOSUM85-60, shown in Using incremental ensemble mountain height from . By Eqs  and 6),,a to thaαseq and an additive shift factor αstr such that the mean and standard deviation for the distribution of sequence similarity values from a RIBOSUM matrix , let pA, pC, pG, pU [resp. a [resp. b], i.e. the proportion of occurrences of each nucleotide A,C,G,U in query a [resp. target b]. Define the mean μseq and standard deviation σseq of RIBOSUM nucleotide similarities byIn order to combine sequence and ensemble structural similarity, we determine a multiplicative scaling factor M matrix   = −d0, where d0 is defined in p  of nucleotide similarity multiplied by αseq is equal to the mean [resp. standard deviation] of structural similarity after addition of shift term αstr:Setting a = a1, …, na and target RNA b = b1, …, mb with incremental ensemble mountain heights ma(1) ⋯ ma(m) of a, mb(1) ⋯ mb(m) of b, and user-defined weight 0 ≤ γ ≤ 1, our final similarity measure is defined byαseq, αstr are computed by Eqs   sequence is the 72 nt tRNA AL671879.2 [resp. 69 nt tRNA D16498.1] with sequence GGGGAUGUAG CUCAGUGGUA GAGCGCAUGC UUCGCAUGUA UGAGGCCCCG GGUUCGAUCC CCGGCAUCUC CA. Then nucleotide query [resp. target] probabilities are (approximately) s in Eqs –14), we, wepA ≈ . By Eqs  and (16)αseq [resp. shift αstr]—recall that αseq and αstr] depend on Ap, Cp, Gp, Up, p of tRNA AL671879.2 and A and similarity measure s, suppose that the expected similarity s is positive for at least one choice of x, y. In the case of BLAST, amino acid and nucleotide similarity scores are integers, for which the Karlin-Altschul algorithm was developed  probabilities Ap, Cp, Gp, Up, p  is equal to (maximum) pairwise sequence similarity of sequences i and j. A rooted tree is then constructed by progressively creating a parent node of the two closest siblings. Parent nodes are profiles (PSSMs) that represent alignments of two or more sequences, hence can be treated as pseudo-sequences in a straightforward adaptation of pairwise alignment to the alignment of profiles. Let’s consider an alignment of N sequences M columns. Let i of the alignment (for 1 ≤ i ≤ M). Suppose p, for x ∈ {A, C, G, U, −}, indicates the probability of occurrence of a nucleotide or gap at column i of alignment A. Then sequence similarity SEQSIM between two columns is defined byAs in CLUSTAL  and the AL Omega , our sof (UPGMA)  and p. 3 (UPGMA) . In UPGMmA(i) denote the arithmetic average of incremental ensemble mountain height at column Aii of sequence The structural measure for a profile is computed from the incremental ensemble heights averaged over each column. Let ai, bi ∈ {A, C, G, U, −}, and the aligned sequences include may contain occurrences of the gap symbol ‘−’, provided that not both Sen) [resp. positive predictive value (PPV)] of a predicted alignment is TP divided by reference alignment length [resp. TP divided by predicted alignment length]. The F1 score is the harmonic mean of sensitivity and PPV, so Sen, PPV, and F1, we consider not only nucleotide pairs , but also include pairs of the form  [resp. ], which represent an insertion in the top sequence of alignment RNAmountAlign, FOLDALIGN, MXSCARNA, etc.). Then a = a1a2a3a4a5, a = ACGUA, b = b1b2b3b4, b = ACUA. The length of alignment N = 6, while BRAliBase 2.1), then sequences a, b are as above, the length of alignment M = 5, while F1 score, Sen, and PPV are duly affected.Sensitivity, positive predictive value, and F1 score for pairwise alignments were computed as follows. Let TPstr is the number of correctly predicted base pairs, FPstr is the number of incorrectly predicted base pairs, TNstr is the number of potential base pairs absent in both predicted and reference structures and FNstr is the number of base pairs in the reference structure that were not predicted.To compare predicted structures with consensus Rfam structures, we computed the Matthews correlation coefficient (MCC)  as follosum-of-pair-scores (SPS)  indicate high [resp. low] structural conservation. All computations made with Vienna RNA Package used version 2.1.7  from each alignment, random sequences a′ [resp. b′] were generated with the same nucleotide frequencies, then a random position was chosen in a′ [resp. b′] in which to insert a [resp. b], thus resulting in a pair of sequences of lengths 4n and 4m. Finally, since sequence identity was at most 70%, the RIBOSUM70-25 similarity matrix was used in RNAmountAlign. Preparation of the benchmarking dataset for local alignment was analogous to the method used in multiple local alignment of [For database , many ofdatabase , yieldindatabase , these anment of .query sequences from the pool of 2220 sequences, 1000 random target sequences of length 400 nt were generated to have the same expected nucleotide frequency as that of the query. For each query and random target, five semiglobal alignments were created using gap initiation costs of gi ∈ {−1, −2, −3, −4, −5} with gap extension cost ge equal to one-third the gap initiation cost. For each alignment score x for query and random target, the p-value was computed as 1 − CDF(x) for ND, EVD and GD, where CDF(x) is the cumulative density function evaluated at x. Additionally, a heuristic p-value was determined by calculating the proportion of alignment scores for given query that exceed x.A pool of 2220 sequences from the Rfam 12.0 database  was creaLocARNA (version 1.8.7), FOLDALIGN (version 2.5), FOLDALIGNM (version 1.0.1), LARA (version 1.3.2), DYNALIGN (from version 5.7 of RNAstructure), the sequence alignment algorithm MUSCLE (version 3.8.31), STRAL (in-house implementation due to unavailability of the software [MXSCARNA (version 2.1). For genome query scan benchmarking, we used RSEARCH 1.1, FOLDALIGN (version 2.5), and RNAmountAlignScan. Recommended default parameters were used for each software, including RNAmountAlign from this paper. The commands used to run the software are given in For benchmarking, we used software , and MXSRNAmountAlignScan, RSEARCH, and FOLDALIGN in genome scanning mode, as described in Section Query Scan, were conducted on a 24-processor Intel Xeon CPU E5-2440 2.40 GHz system with 198GB memory.In contrast to all other benchmarking work described in this paper, benchmarking tests for RNAmountAlign’s performance for pairwise and multiple alignments on BRAliBase K2 and K5 datasets, respectively.We benchmarked pairwise global alignment sensitivity, positive predictive value (PPV) and F1-score for the software described in this paper, as well as for LocARNA, FOLDALIGN, LARA, DYNALIGN, STRAL, and MXSCARNA. For pairwise benchmarking, reference alignments of size 2, a.k.a. K2, were taken from the BRAliBase 2.1 database [BRAliBase 2.1 K2 data are based on seed alignments of the Rfam 7.0 database, and consist of 8,976 alignments of RNA sequences from 36 Rfam families.database . BRAliBaBRAliBase 2.1 data . In computing moving averages, each value represents the average over a symmetric window  of size 11 nt centered at the value from the x-axis. For 504 pairwise global alignments, FOLDALIGN produced “No global alignment was found. This can either be due to the pruning, or because no structural alignment exists” and for 29, DYNALIGN yielded no alignments. The only BRAliBase alignment that both tools failed was Hammerhead_1.apsi36.sci72.no1. In our benchmarking the sensitivity, PPV and F1 values for such failures of FOLDALIGN and DYNALIGN were all assigned the value of 0, which explains the weaker performance of these two methods in FOLDALIGN 2.5 than in the previous work of [RNAmountAlign, gap initiation cost was -3, gap extension -1, and sequence/structure weighting parameter γ was 0.5 . The sequence-only alignment is computed from RNAmountAlign with the same gap penalties, but for γ = 0. While its accuracy is high, RNAmountAlign is faster by an order of magnitude than LocARNA, LARA, FOLDALIGN, and DYNALIGN—indeed, algorithmic time complexity of our method is O(n3) for two sequences of length n. Since STRAL could not be compiled on any of our systems, we implemented its algorithm by modifying RNAmountAlign and obtained results for STRAL’s default parameter settings. Therefore, the run time of STRAL is identical to RNAmountAlign but RNAmountAlign achieves higher F1 score, sensitivity and PPV. MXSCARNA and RNAmountAlign have similar average F1 scores. Indeed, a paired 2-tailed Wilcoxson signed-rank test of the difference between F1 scores, as computed by MXSCARNA and RNAmountAlign, for the 8,976 pairwise global alignments mentioned in H0 cannot be rejected, where H0 asserts that the difference is 0 between F1 scores of MXSCARNA and RNAmountAlign—see MXSCARNA has slower run time than RNAmountAlign. Both MXSCARNA and RNAmountAlign support global and local alignment; however, unlike MXSCARNA, RNAmountAlign also supports semiglobal alignment and reports p-values. The right panel of RNAmountAlign, with the next fastest software, MXSCARNA and LARA. Unlike the graph in the left panel, actual run times are shown, graphed as a function of sequence length, rather than logarithms of moving averages.Moving averages (window size 11) of sensitivity, positive predictive value, and F1 score were computed as a function of sequence identity, where it should be noted that the number of pairwise alignments for different values of sequence identity can vary for the  work of . DefaultRNAmountAlign, LocARNA, LARA, FOLDALIGN, DYNALIGN, STRAL, and MXSCARNA when benchmarked on 36 families from the BRaliBase K2 database comprising altogether 8,976 RNA sequences with average length of 249.33. Averaging over all sequences, the F1 scores for the programs just mentioned were respectively 0.8370, 0.7808, 0.8406, 0.7977, 0.6822, 0.8247, 0.8402; i.e. F1 score 0.8406 of LARA and 0.8402 of MXSCARNA slightly exceeded the F1 score 0.8370 of RNAmountAlign and 0.8247 of STRAL, while other methods trailed by several percentage points. In addition, RNAmountAlign, FOLDALIGN, and LocARNA. We had intended to include SCARNA_LM [SCARNA_LM no longer appears to be maintained . Since the reference alignments for the local benchmarking dataset are not known, and sensitivity depends upon the length of the reference alignment, we only report local alignment length and positive predictive value. Abbreviating RNAmountAlign by MA, FOLDALIGN by FA, and LocARNA by LOC, p-values for a 2-tailed paired Wilcoxon signed-rank test whether the difference in positive predictive values fo 1,500 pairwise local alignments.Although there appears to be no universally accepted criterion for quality of local alignments, CARNA_LM  in the bRNAmountAlign has comparable accuracy, but much faster run time, hence making it a potentially useful tool for genome processing applications. Here it should be stressed that all benchmarking results used equally weighted contributions of sequence and ensemble structural similarity; i.e. parameter γ = 1/2 when computing similarity by Taken together, these results suggest that p-values for semiglobal alignment scores between Rfam sequences and random RNA. As explained earlier, a pool of 2220 sequences from the Rfam 12.0 database [gi = −5, −4, −3, −2, −1 with z for query and random target, the p-value was computed as 1 minus the cumulative density function, 1 − CDF(z), for fitted normal (ND), extreme value (EVD) and gamma (GD) distributions, thus defining 1000 p-values. Additionally, a heuristic p-value was determined by calculating the proportion of alignment scores for given query that exceed z. For each set of 2.5 million (500 × 5 × 1000) p-values , Pearson correlation values were computed and displayed in the upper triangular portion of m ⋅ column + b, where column values constitute the independent variable. Assuming that heuristic p-values constitute the reference standard, it follows that p-values computed from the normal distribution correlate best with semiglobal alignment scores computed by RNAmountAlign. Earlier studies have suggested that protein global alignment similarity scores using PAM120, PAM250, BLOSUM50, and BLOSUM62 matrices appear to be fit best by the gamma distribution (GD) [RNAmountAlign semiglobal alignment depends on the gap costs applied . Since there is no mathematical theory concerning alignment score distribution for global or semiglobal alignments, it must be up to the user to decide which distribution provides the most reasonable p-values.database  was creaion (GD) , and thaRNAmountAlign with the software LARA, mLocARNA, FOLDALIGNM, Multilign, MXSCARNA, and sequence only MUSCLE for multiple global K5 alignments in BRAliBase 2.1. STRAL is not included since the source code could not be compiled. -sci flag of RNAalifold to compute SCI from the output of each software without reference to the reference alignment. RNAmountAlign produces SPS scores comparable to mLocARNA, LARA and MXSCARNA, and higher than Multilign and FOLDALIGNM while the SCI score obtained from RNAmountAlign are lower than other software. Averaging over all sequences, the SPS scores for RNAmountAlign, LARA, mLocARNA, FOLDALIGNM, Multilign, MXSCARNA, and MUSCLE were respectively: 0.81 ± 0.18, 0.85 ± 0.15, 0.86 ± 0.15, 0.74 ± 0.24, 0.63 ± 0.17, 0.84 ± 0.15, and 0.82 ± 0.17, while SCI scores are respectively: 0.84 ± 0.24, 0.92 ± 0.22, 0.96 ± 0.21, 0.88 ± 0.23, 0.96 ± 0.21, 0.91 ± 0.20, and 0.79 ± 0.26. SPS score for each reference alignments is 1 by definition, and average SCI score over all reference alignments is 0.79 ± 0.26. RNAmountAlign has faster run time than all other structural alignment software in our benchmarking tests, thus confirming the earlier result from pairwise benchmarking.We benchmarked RNAmountAlign, called RNAmountAlignScan, which scans a target genome sequence to find hits having high sequence and structural similarity to a given query sequence. RNAmountAlignScan functions by computing optimal semiglobal alignments of the query to sliding windows of the target, then returns the aligned target segments sorted by p-value. For a query of length n, target genome of length m, window size w > n, and window step size of s, a total of RNAmountAlignScanRNAmountAlignScan, we compared it with RSEARCH [FOLDALIGN and RNAmountAlignScansequence-only where only sequence homology is considered. RSEARCH takes a single query sequence with its secondary structure and performs local alignment with the target genome, in order to search for homologous RNA. For our comparison, we used the query sequence from E. coli K12 MG1655 genome having 4, 641, 652 nt. Since tRNAscan-SE [tRNAscan-SE. tRNAscan-SE found 49 full tRNA sequences on the forward strand and 37 on the reverse strand. Default parameters of each software package were used to search the query in the forward strand of E. coli genome  without such memory errors. RSEARCH is not shown in the plot because it reported 89 hits, none of which had any overlap with tRNAs predicted by tRNAscan-SE and thus its sensitivity and PPV are equal to zero. Sequence length of the predictions returned by RSEARCH was 11.73 ± 0.58. For calculation of PPV as a function of sensitivity, for each of the three methods on each strand, we sorted the predictions in descending order by p-value. Given the i-th prediction, we determined the number of predictions preceding the current i-th prediction (including the i-th), and calculated the number of true positives in that collection then divided this number by 49 for positive or 37 for reverse strands to obtain sensitivity. True positives are predictions with overlap proportion of greater than 0.8 to any of the tRNAscan-SE sequences. The overlap proportion between predictions and tRNAscan-SE is defined as the ratio |A ∩ B|/|B|, where A is a tRNA predicted by each software and B is a tRNA detected by tRNAscan-SE, the latter assumed to be the gold standard. FOLDALIGN with area under the curve of 0.90 and 0.64 for forward and reverse strands outperforms RNAmountAlignScan with 0.27 and 0.28 and sequence-only with 0.08 and 0.16. However, only one the FOLDALIGN predictions on each strand had p-value less than 0.1 while RNAmountAlignScan reported significant p-values for all of the true positives FOLDALIGN finished in 12.06 hours while run time of RNAmountAlign was 1.63.We developed an extension of  RSEARCH , FOLDALInome see ; in the false positive of RNAmountAlignScan; i.e. that prediction having statistically significant sequence and structural similarity to the query tRNA, but which shares no overlap with a tRNA predicted by tRNAscan-SE. This prediction has 42% sequence identity to the query tRNA, and its MFE forms a cloverleaf secondary structure. In contrast to all other benchmarking computations of this paper, due to the memory requirements for genome scanning mode, each program for query search was run on a 24-processor Intel Xeon CPU E5-2440 2.40 GHz system with 198GB memory. The commands to run the software are given in FOLDALIGN, RNAmountAlignScan, and RSEARCH were respectively 12.06, 1.63, and 0.62 hours and memory usage were 0.57, 20.61 and 0.49 Gigabytes. The number of random RNAs used for p-value computation in both RSEARCH and RNAmountAlignScan was 1000, and both used the RIBOSUM 85-60 matrix. The outputs of all the program are available at http://bioinformatics.bc.edu/clotelab/RNAmountAlign/.RNAmountAlign is a new C++ software package for RNA local, global, and semiglobal sequence/structure alignment, which provides accuracy comparable with that of a number of widely used programs, but provides much faster run time. RNAmountAlign additionally computes E-values for local alignments, using Karlin-Altschul statistics, as well as p-values for normal, extreme value and gamma distributions by parameter fitting.S1 Table(TIFF)Click here for additional data file.S2 Table(TIFF)Click here for additional data file.S3 Table(TIFF)Click here for additional data file.S4 Tablep-values computed separately for each family are available at http://bioinformatics.bc.edu/clotelab/RNAmountAlign.Tables of (TIFF)Click here for additional data file.S5 Tablep-values computed separately for each family are available at http://bioinformatics.bc.edu/clotelab/RNAmountAlign.Tables of (TIFF)Click here for additional data file.S6 TableRNAmountAlign, LocARNA, LARA, FOLDALIGN, DYNALIGN, STRAL and MXSCARNA are listed, along with pooled averages over all 8,976 pairwise alignments. Parameters used in RNAmountAlign were similarity matrix RIBOSUM85-60, structural similarity weight γ = 1/2, gap initiation gi = −3, gap extension ge = −1.For each indicated Rfam family, the the number of alignments (NumAln), sequence identity (SeqId), and sensitivity scores for (TIFF)Click here for additional data file.S7 TableRNAmountAlign, LocARNA, LARA, FOLDALIGN, DYNALIGN, STRAL and MXSCARNA are listed, along with Pooled averages over all 8,976 pairwise alignments. Parameters used in RNAmountAlign were similarity matrix RIBOSUM85-60, structural similarity weight γ = 1/2, gap initiation gi = −3, gap extension ge = −1.For each indicated Rfam family, the the number of alignments (NumAln), sequence identity (SeqId), and PPV-scores for (TIFF)Click here for additional data file.S8 TableAp, Cp, Gp, Up, a total of N = 10000 RNA sequences of length n = 200 were randomly generated to have the given expected nucleotide frequency. To compute p were determined.The full table (not shown) has 1770 rows. To determine average base pairing probabilities, given nucleotide probabilities  Package  was made(TIFF)Click here for additional data file.S1 Figalifold from Vienna RNA Package. The figure is obtained from RNAalifold web server.The consensus structure is computed by a calling function (TIFF)Click here for additional data file.S2 FigBRAliBase 2.1 database used for benchmarking. Moving averages taken for centered, symmetric windows of size 11.Moving averages of F1-score for centered, symmetric windows of size 11 are shown as a function of sequence identity for pairwise alignments in the (TIFF)Click here for additional data file.S3 FigBRAliBase 2.1 database used for benchmarking. Moving averages taken for centered, symmetric windows of size 11.Sensitivity is shown as a function of sequence identity for pairwise alignments in the (TIFF)Click here for additional data file.S4 FigSen and PPV, pairs of the form  and  are also counted while SPS is the average pairwise sensitivity only considering aligned residue pairs (Note that in our definition of ue pairs . However(TIFF)Click here for additional data file.S5 FigBRAliBase 2.1. Overall average MCC values ± one standard deviation are shown in parentheses. These 7, 154 alignments were selected with the property that both sequences in the alignment (exactly) appear in an Rfam family seed multiple alignment from Rfam 7.0, and so can be assigned an Rfam consensus secondary structure as described in the text. These consensus structures are taken as the reference structures in the computation of MCC. Predicted structures are obtained directly from the output of each software in the benchmarking test. For RNAmountAlign and for our in-house implementation of STRAL, the -alifold flag was used to compute the consensus structure by a function call to alifold from libRNA.a in the Vienna RNA Package.Moving averages of MCC are computed for centered, symmetric windows of size 11 and shown as a function of sequence identity for a subset of 7, 154 reference pairwise alignments from k2 (TIFF)Click here for additional data file.S6 FigBRAliBase 2.1 database used for benchmarking. Moving averages taken for centered, symmetric windows of size 11. This figure should be compared with Moving averages of F1-score for centered, symmetric windows of size 11 are shown as a function of sequence identity for pairwise alignments in the (TIFF)Click here for additional data file.S7 FigBRAliBase 2.1 database used for benchmarking. Moving averages taken for centered, symmetric windows of size 11. This figure should be compared with Sensitivity is shown as a function of sequence identity for pairwise alignments in the (TIFF)Click here for additional data file.S8 FigRNAmountAlign genome-scanning software, semiglobal alignments of the query tRNA AB031215.1/9125-9195 were made with each 300 nt window (successive window overlap of 200 nt) of the E. coli str. K-12 substr. MG1655 genome. This figure shows the MFE structure, color-coded by positional entropy [p-value of 4.145010 ⋅ 10−6. . However, this RNA is clearly not a tRNA, since the three loops are not within the scope of a multiloop, and the variable loop is located in the wrong position, and the large positional entropy suggests that there is not an unambiguous structure. Moreover, this sequence is not one of the 40 tRNA genes/pseudogenes on the plus-strand predicted by tRNAscan-SE [Using  entropy , for theAscan-SE .(TIFF)Click here for additional data file.S1 Appendix(PDF)Click here for additional data file.


Many genetic variants have been reported from sequencing projects due to decreasing experimental costs. Compared to the current typical paradigm, read mapping incorporating existing variants can improve the performance of subsequent analysis. This method is supposed to map sequencing reads efficiently to a graphical index with a reference genome and known variation to increase alignment quality and variant calling accuracy. However, storing and indexing various types of variation require costly RAM space.Aligning reads to a graph model-based index including the whole set of variants is ultimately an NP-hard problem in theory. Here, we propose a variation-aware read alignment algorithm (VARA), which generates the alignment between read and multiple genomic sequences simultaneously utilizing the schema of the Landau-Vishkin algorithm. VARA dynamically extracts regional variants to construct a pseudo tree-based structure on-the-fly for seed extension without loading the whole genome variation into memory space.We developed the novel high-throughput sequencing read aligner deBGA-VARA by integrating VARA into deBGA. The deBGA-VARA is benchmarked both on simulated reads and the NA12878 sequencing dataset. The experimental results demonstrate that read alignment incorporating genetic variation knowledge can achieve high sensitivity and accuracy.https://github.com/hitbc/deBGA-VARA.Due to its efficiency, VARA provides a promising solution for further improvement of variant calling while maintaining small memory footprints. The deBGA-VARA is available at:  An accurate and complete understanding of genetic variation is important in research on human disease –3. A funThese large genetic variations are likely to lead to lots of unmapped reads or low-quality mapped reads, resulting in poor quality on characterization of individual genomes and coverage fluctuations . MoreoveDefining new data structures that can repersent different kinds of genome variants can be very challenging. Reads mapping to de Bruijn graphs, according to Limasset et al. , was sugFurthermore, Most of the state-of-the-art generic aligners are implemented in seed-and-extension strategy. The seed extension as a compute-intensive step accommodates the alignment of the read to local sequences surrounding each candidate seed to determine the most likely read position. When determining optimal alignments, approximate string matching in local extension can be crucial, especially for variation graph-based models. The Smith-Waterman (SW) algorithm , 31 affehttps://github.com/hitbc/deBGA-VARA).In order to achieve a lower RAM usage during our dynamically construction of pseudo-tree based structure variation tree with different genomic sequence and regional indeded varians in the process of seed extension, we did not load the whole variation set. Also, we developed a read-variation tree alignment algorithm, VARA, utilizing the Landau-Vishkin algorithm and breadth-first traversal on the paths that may consist of the tree nodes and their corresponding variation. Comparing other strategies for tree index-based alignment with reads and a significant number of sequence in the same time, this method can be more efficient and effitive. We intergated VARA into deBGA, a de Bruijn graph-based read aligner, we were able to implement a more comprehensive mapping tool, deBGA-VARA  collect local genomic sequences around the candidate seeds that are generated from the deBGA seeding phase.2) assemble all the corresponding variant knowledge and construct the variation tree for the novel seed extension.3) variation-aware alignment between variation tree and the segmental read to generate the candidate paths.4) optimal alignment selection and output the read mapping result.During genome indexing, the knowledge of INDELs, including variant sequence, variant type, the length and its location, is stored in independent indexing files (ALT-seq) from the original genome index. Also, we intergrated the SNP information into the reference sequence in order to form a variation-aware reference encoded in 4-bit format. In the format mentioned before, both genomic character and its corresponding SNPs can be recorded at one site. During seed extension, to perform alignment with a combination of regional variants, we constructed a pseudo tree-based structure variation tree on-the-fly for indexing genomic sequences and known variation.The sequences for extension are likely to have identical fragments due to so many repetitive regions in human reference. The variation tree collects all local sequences used to construct the tree structure for simultaneous alignment. The nodes in the tree are created alphabetically using the identical part of two or more sequences. One specific node is able to have multiple successor nodes that are located at the subsequent branches, and the node at its next layer is the adjacent successor node. Also, the variants in ALT-seq can connect to the tree node according to its genomic positional relationship to each sequence. Thus, there can be multiple insertions with connections to the same site of an identical node. The long deletion variant can stretch over several nodes, and it is possible to finish at a location beyond the sequence length. Under such circumstance, a new pseudo tree node needs to be created. As shown in Fig. After building the tree, a path can be generated from a traversal from one node to its successor node or the variant on the way. All possible paths are enumerated in advance according to sequences and correlations between variations and tree nodes. Notably, all paths are independent from each other with a unique ID. There cannot be more than one traversal of identical sequences during subsequent alignment, and some of the nodes can never be reached because of early termination of path traversal due to mismatch occurrence.Pa is a array that records the path ID; ii) Na is a array that records the corresponding nodes on the current path; and iii) Pka is an array that records cadidate paths. A mismatch-tolerant string comparison strategy was also performed by this method, i.e., it defines an exact match if one character of a read can match a genomic base or SNP at the same locus of reference. Hence, with various types of variation, VATR can accommodate read alignment to variation trees.We designed the variation tree-based traversal algorithm VATR for breadth-first traversal of all paths, as shown in Algorithm 1. This method adopts a queue with its basic operations to align a read to a path starting with an arbitrary node. The whole procedure is majorly classified into three categories: i) straightforward node sequence alignment without any variation; ii) traversing nodes including variants and path ID computation; and iii) read exact alignment with a variant sequence (insertion). More specifically, there are also three types of data about to enter the queue: i) the adjacent successors of the current node; ii) newly reached variants; and iii) the ending node of the current variant. Herein, i) O(en) time complexity, where e is the maximum edit distance and n is the length of the sequence. This method considered the fact that it is not necessary that the algorithm tries to expand the computational cells in a typical dynamic programming matrix (DPM) with an editing distance greater than the threshold. Herein, DPM is the cell on diagonal d of DPM, such that j−i=d, where i is the row and j is column of the matrix. Meanwhile, it maintains an editing distance matrix LVM, which stores the longest matching distance along current diagonal d with edit distance e , insertion and deletion. Landau-Vishkin iteratively calculates cells of LVM to exploit optimal alignment with the longest matches, and its recurrence relations are as follows:The Landau-Vishkin algorithm improved the dynamic programming strategy over conventional banded global alignment methods and achieved string matching in e e Fig. a. HereinPs is the position query read starts. Based on this principle, we developed a novel global alignment algorithm, VARA, by embedding the traversal alignment process for each path on the variation tree in the Landau-Vishkin framework, as shown in Algorithm 2. The following vital structures store temporary results and contribute to the connectivity of each iteration of traversal alignment.where Pas denotes two-dimensional arrays that record the ID set of paths. More specifically, Pas[d] represents the set of paths corresponding to current variation type d.i) Nas is a three-dimensional array that records the ID set of nodes, and Nas[e][d] represents the set of nodes corresponding to different variant types d under different edit distances e.ii) Pkas is a two-dimensional array that keeps a record of the ID set of the paths, and Pkas[e] represents the set of paths under current edit distances e.iii) LVM to a three-dimensional matrix (L) for storage of variation types and editing distances for multiple paths and continuously updated L in VATR for newly generated paths. As shown in Fig. LVM, each cell records the paths set for current edit distance e with the variation d  for different variant types during the current traversal process . For a specific upper limit of edit distance e, d denotes edit distance within the range from −e to e, and the positive value, negative value and zero represent deletion(D), insertion(I) and mismatch(M), respectively. This example shows the traversing procedure from the alignment with distance e =1 to e =2, and the edges in the figure represent the traversal relationship between paths. For instance, path2 in the previous period continues to traverse to path2 and path4 for three types of variations. We can see that traversals from different ancient paths can reach the path with the same ID, and the matrix on the right side is given as a summary of involved paths for different variation types in the current period, e.g., path1 is only reached with the deletion type, and path3 can be reached with both mismatch and insertion types. After the first round of iteration, the new pset had path1, path2, path3, path4 and path5. After the second round for distance e =2, the new SET and pset were generated in the same way. This pset continued to update as the edit distance increased via rounds of iterations. By this means, we collected all candidate alignments of traversed paths before the end of the entire process.We provide a detailed description of the flowchart of the VARA algorithm  on the same path. The threshold was based on the fact that it is less likely for a read with a length of 100-300 bps to contain several structural variations simultaneously.We developed the novel mapping tool deBGA-VARA through integrating VARA into the de Bruijn graph-based aligner deBGA in the phase of extension to accomplish alignment with variation knowledge, only if there are no valid matched seeds for paired-end reads, mainly due to variant occurrence. To be specific, two sides of each candidate seed are semiglobal aligned. For example, we collected all sequences of different regions for the left and right sides of the current seed. After the collection procedure, we used regional variants to built its corresponding variation tree. For seeds only found in a single end, we anchored them to the other end by insertion distance utilizing the strategy in primary deBGA extension. Herein, all seeds from the anchored side underwent VARA analysis except for the seed with the maximum matching length. After such implementations of each end, we merged the outcomes according to their positions and selected the optimal alignment.Because of the alignment containing variation, restoring CIGAR to repersent the matching operations in both the read and the original reference sequence was considered to be necessary. We combined the path sequence including the reference and variant knowledge to generate the primary reference-based cigar result. As shown in Fig. 1) The reference cigar is a mismatch or match if the ALT cigar is 1bp deletion in the context of the 1bp insertion variation.2) The reference cigar is a mismatch or match if the ALT cigar is 1bp insertion in the context of the 1bp deletion variation.3) The reference cigar is an insertion if there exists a deletion in the ALT cigar in the context of the insertion variation with the length of more than 1bp.4) There exists deletion and mismatch in the reference cigar if there exists a insertion in the ALT cigar in the context of the deletion variation with the length of more than 1bp.Moreover, for each match in the ALT cigar, it needs to restore the cigar according to its corresponding SNP and genomic base.The deBGA-VARA was evaluated on two simulation datasets and an HTS dataset to assess its accuracy, sensitivity and speed. Four state-of-the-art aligners, deBGA, BWA-MEM, BWBBLE and vg, were employed for comparison. Herein, BWBBLE was accessed with multiple maximum numbers of mismatches and gaps in the alignment (-n option in the software). All the benchmarks were conducted on a server with 2 Intel E5-2630v3 CPUs at 2.4 GHz , 512 GB RAM and 48 TB hard disk space (7200 rpm RAID SAS hard disk drive with XFS File System). A single CPU thread (Time-t1) and 8 CPU threads (Time-t8) were used In the benchmarking. The runtime of index loading was excluded for all benchmarked methods.In order to figure out the performance of the startegy to align reads to regions with different variations, we benchmarked deBGA-VARA with soft clipping statistics. Soft clipping that is represented by character S in the CIGAR specification often appears in the alignment results, indicating the read is not properly aligned to the reference; this is often caused by sequencing errors and various types of variation.We collected all the variation knowledge of the individual sample NA12878 in Variant Call format (VCF)  files thThe results of the simulation dataset are shown in Table i) The deBGA-VARA is several times faster than BWA-MEM, BWBBLE and vg. For example, it is on average fourfold as fast as BWA-MEM. The BWBBLE is almost 250 times slower than deBGA-VARA, even when it is configured to achieve a comparable higher accuracy . Meanwhile, deBGA-VARA and vg achieved almost 100% accuracy for Sim-i100 and Sim-i250; while the accuracy of BWBBLE dropped from 95.2% to 84.4%, even with a large allowable maximum difference (n =10) with the growth of read length. The deBGA-VARA has fewer unmapped reads than deBGA, BWA-MEM and BWBBLE but more than vg, mainly due to that some large known variants are integrated in the seeding phase for vg alignment. Future work should integrate large and complex variants into the seed collection to further increase accuracy and sensitivity. The deBGA-VARA has almost as many unmapped reads as deBGA, mainly because both have identical seeding strategies.ii) The deBGA-VARA has fewer soft clipping reads than all other methods except BWBBLE (there is no clipping read output of BWBBLE), i.e., the number of soft clipping reads in BWA-MEM is several orders of magnitude larger than that of deBGA-VARA. This finding indicates that reads mapping with known genetic variation can effectively improve the alignment quality. Moreover, there are more clipping reads in vg as the read length increases; however, the quantity of clipping reads in deBGA-VARA decreases greatly.iii) Vg often requires large amounts of space and time to construct the index, i.e., it requires approximately 320 GB RAM and 20 TB disk space (-X =3 option in the software) to build the xg and gcsa2 index of the human reference genome (GRCh37/hg19) and 200 MB VCF dataset in approximately 40 hours (16 CPU threads). Meanwhile, the deBGA-VARA and BWBBLE require fewer resources to index this identical reference and variation dataset, e.g., 40 GB RAM and 27 mins for BWBBLE and 30 GB RAM and 4 hours for deBGA-VARA. Herein, the index used in the VARA algorithm only requires 2.5 GB memory footprints in alignment.Compared to conventional methods, global alignment with known variation can improve the mapping quality and accuracy. To prove the scalability of deBGA-VARA, we showed six specific examples of read alignments for all aligners with various types of variation in the simulation and HTS datasets Fig. . In eachWe also benchmarked deBGA-VARA on the sequencing dataset of the sample NA12878 that was sequenced by Illumina HiSeq 2000 . As shown in Table There are more clipping reads than those in the simulated data set, mainly due to the many sequencing errors and various unknown variation information. A priori variation can contribute to reducing soft clipping reads, i.e., deBGA-VARA has the minimum number of soft clipping reads in the whole genome. Hence, deBGA-VARA maintains high accuracy and sensitivity while reducing soft clipping reads, both on a simulated and sequencing dataset. This method also shows more advantages over both typical HTS read aligners and current variation-aware alignment methods.Known variation can contribute to determining the correct result but not arbitrary selection of one of the optimal alignments. As a result, this factor can effectively reduce false positives for novel variant discovery. For the known small deletions (3 bp) in the HTS dataset Fig. b, deBGA-We further benchmarked the read alignments produced by deBGA-VARA as well as other state-of-the-art aligners in MHC region. The 4.6-MB extended MHC region (chr6: 28477797-33448354) in human genome is highly reputed by its dense and complex variations, i.e., genomic variations more frequently occur in this region and the combinations of the variations are divergent for various samples. Previous studies  have demWe evaluated the alignment results in MHC region by two aspects. Firstly, we assessed the number of seed hits of the reads to this region. This is critical that, for a given read, the alignment may fail if the aligner cannot find the seed(s) successfully hit the correct locus. We found that the number of correct hits in the reads from MHC region is to some extent lower than that of the reads from the regions having less variations. However, there are still enough hits for them to recognize the correct candidate regions to implement extension alignment, although there are dense variations. Two examples are shown in Fig. Furthermore, we assessed results of base-level local alignment during the extension phase, which are shown in Table Local and global alignments play a fundamental role in HTS read mapping and downstream sequence analysis, i.e., the CIGAR outcome of the alignment contributes to variant calling and structural variation detection. However, there is still a high demand to decrease the number of false positive novel variants due to the incorrect alignment results from current mapping methods. Sequences alignment with existing variation can provide a novel strategy to further improve alignment accuracy and mapping quality. We found that integrating the whole set of existing variants into the reference results in explosive growth of graph size in a typical variant graph paradigm. It is non-trivial to handle graph construction and traversal with a large quantity of variation because the possible path enumeration can theoretically be an NP-hard problem. Furthermore, this method often has exponential time and space complexity to reconstruct the index for the continuously updating variation.Herein, we propose a novel global alignment algorithm, VARA, and developed the mapping system deBGA-VARA by integrating it into a deBGA aligner. We regard deBGA-VARA as a lightweight variant graph-based mapping algorithm. This algorithm combines known variation only to global alignment in the extension step and dynamically constructs pseudo tree-based structures to index variants and sequences in a local genomic scope. The method can be memory scalable due to its limitation of possible variants and paths in VARA. This characteristic can be very beneficial to aligning reads to large variation-aware references. Moreover, the benchmarking results on the simulation and sequencing datasets demonstrated that deBGA-VARA runs much faster than state-of-the-art approaches while maintaining higher sensitivity and accuracy. With its scalability, deBGA-VARA can achieve highly confident alignments both in the whole genome and MHC region. Seeding with variation cannot significantly improve the alignment quality but decreases the mapping efficiency. The deBGA-VARA also showed better results with the increase in read length, indicating its potential for forthcoming sequence analysis.It is necessary to integrate large structural variations, e.g., long deletions and insertions, into seed exploration and merging to furtherly increase the quantity of true candidate seeds. Other studies should use this approach in a recurrent framework that will utilize current novel variation outcomes as the input prior knowledge of the next loop of alignment in deBGA-VARA. This strategy can keep improving the quality of novel variants in this iterative process until convergence.Overall, deBGA-VARA is a promising tool for variation-aware read alignment. This method shows enormous potential in variant calling and complex variation detection for a large population of genomes.