We are pursuing the hypothesis that visual exploration and learning in young infants is achieved by producing gaze-sample sequences that are sequentially predictable. Our recent analysis of infants’ gaze patterns during image free-viewing  provides The capacity to perceive and recognize objects begins to develop shortly after birth e.g., . A critiIn the current paper, we present and evaluate a microanalytic approach for analyzing infants’ gaze shift sequences during visual exploration. Specifically, we convert the sequence of fixations produced by each infant into a stream of “center-of-gaze” (or COG) image samples, where each sample approximates the portion of the image visible to the fovea of a human observer while fixating the given location on the image  that were presented with the same natural images . The reaperceptual-completion task, which is specifically designed to assess infants’ perception of a moving, partially occluded object . Because our focus here is on infants’ initial gaze patterns at the beginning of the task, before they have accumulated extensive experience with the display, we therefore restrict our analyses to gaze data from the first trial of the occluded-rod display. Although this display is somewhat simplified relative to the natural images from our previous study, it also has the benefit that infants will likely devote much of their attention to either of the two primary objects in the scene , thereby producing a rich source of object-directed gaze data to analyze.In the current paper, we extended our model in a number of important ways to investigate the development of object perception in 3-month-olds. First, our dataset derives from a paradigm called the d object . Figure A second important advance in the current paper concerns how the artificial-observer gaze patterns are produced. Specifically, in our previous model, several parameters of the artificial observers were left to vary freely, which resulted in systematic differences between the kinematics of the gaze patterns produced by the human-infant and artificial observers. For example, the artificial observers generated significantly longer gaze shifts than the infants. We address this issue in the current model by carefully yoking the gaze patterns of each artificial observer to a corresponding individual infant, so that the average kinematic measures were the same for each observer group.intrinsically motivated learner . However, because the issue of intrinsic motivation is not central to the current paper, we have stripped this component from the model, resulting in a more direct and straightforward method for assessing the relative learnability of the COG sequences produced by each of the observer groups.A third advance is that we also simplified the architecture of the model used to learn the COG sequences. In particular, our previous model focused specifically on the process of visual exploration, including a component in the model that simulated an In the next section, we provide a detailed description of (1) the procedure used to transform infants’ gaze data into COG sequences, (2) the comparable steps used to generate the artificial observers’ gaze data and COG sequences, and (3) the training regime employed to measure COG sequence learnability. In the meantime, we briefly sketch the procedure here, followed by our primary hypotheses and analytical strategy.Figure 1A. Fixation locations for each infant were acquired by an automated eye-tracker. These locations were then mapped to the corresponding spatial position and frame number from the occluded-rod display, and a small (41 × 41 pixel) image sample, centered at the fixation location, was obtained for each gaze point. Next, two sets of artificial gaze sequences were generated. First, an image-saliency model was used to produce a sequence of gaze points in which gaze direction is determined by bottom-up visual features, such as motion or regions with strong light/dark contrast , but also that this difference would persist throughout training.Given our previous findings with the image free-viewing paradigm, our primary hypothesis was that the COG sequences produced by infants during the occluded-rod display would be more easily learned by a set of SRNs than either of the two artificial-observer sequences. We evaluated this hypothesis by assigning an SRN to each of the infants, and then training each network simultaneously on the three corresponding COG sequences . Learning was implemented in each SRN by presenting it with the three corresponding COG sequences, one image sample at a time as input, and then using a supervised learning algorithm to train the SRN to produce as output the next image sample from the sequence. We then assessed In addition, we also probed the training process further by exploring the effect of manipulating the context units on the performance of the SRN. In particular, we implemented a “forgetting function” in which the context units were reset at one of three intervals . For example, in the 5-sample case, the first COG sample in a wave of five is effectively presented to the network as input (directly or indirectly) four times: once as the first COG sample, and then four more times as the trace of the sample cycles through the context units. By this logic, the fourth COG sample in the same wave of five is presented twice. Thus, the forgetting function provides a somewhat qualitative method for revealing whether or not sequential or temporal structure is present in infants’ COG image samples, but may not directly specify how those regularities are distributed over time. We return to this issue in the discussion and raise a potential strategy for addressing it.During the collection of eye-tracking data (see below), the occluded-rod display was rendered in real-time. In order to convert this display into a sequence of still frames for the current simulation study, it was first captured as a video file , and then parsed by Matlab into still frames. A complete cycle of the rod’s movement, from the starting position on the far right, to the far left, and then back to the starting location, was extracted from the video and resulted in 117 frames . Note that during video presentation, the dimensions of the occluded-rod display were 480 × 360 pixels, which was presented at the center of the monitor, surrounded by a black border. This border was subsequently cropped from the still-frame images, so that the occluded-rod display filled the frame. The gaze data obtained from infants were adjusted to reflect this cropping process; meanwhile, as we describe below, the simulated gaze data from the image-saliency and random-gaze models were obtained by presenting the cropped (480 × 360) occluded-rod displays to each model.M = 87.7 days, SD = 12 days; 5 females) participated in the study. Infants sat on their parents’ laps approximately 60 cm away from a 76 cm monitor in a darkened room. Eye movements were recorded using the Tobii 1750 remote eye tracker. Before the beginning of each trial, an attention-getter (an expanding and contracting children’s toy) was used to attract infants’ gaze to the center of the screen. As soon as infants fixated the screen, the attention-getter was replaced with the experimental stimulus and timing of trials began. Each trial ended when the infant looked away for 2 s or when 60 s had elapsed. Note that all analyses described below were based on the eye-tracking data acquired during each infant’s first habituation trial .Twelve 3-month-old infants  from each still-frame image, and then summing the feature maps into a saliency map. The sequence of 117 saliency maps was then used to generate a series of simulated fixations. We describe each of these processing steps in detail below.Feature maps. Each of the still-frame images was passed through a bank of image filters, resulting in four sets of feature maps: one motion map , four oriented edge maps , one luminance map, and two color-contrast maps . In addition, this process was performed over three spatial scales , by successively blurring the original image and then repeating the filtering process .The second column presents the mean saliency “captured” by each model, that is, the degree to which each group’s fixations were oriented toward regions of maximal saliency in the display. This was computed by projecting the gaze points produced by each of the observer groups on to the corresponding saliency maps, and then calculating the average saliency for those locations. Recall that values on the saliency maps were scaled between 0 and 1; the average saliency values from each group therefore reflected the proportion of optimal or maximal saliency captured by that group. There are two key results. First, the saliency model achieved an average of 0.65 saliency, indicating that – due to the constraint imposed on allowable gaze-shift distance – the model did not consistently fixate the most salient locations in the display. The second noteworthy finding is that infants’ gaze patterns captured a comparable level of saliency, that is, 0.66. As most frequently visited (as estimated by the 41 × 41 fixation window). The maximum value was then averaged across observers within each group, providing a metric for the peak proportion of fixations that a particular location in the occluded-rod display was visited, on average. As Table 1 illustrates, a key finding from this analysis is that infants had the highest revisit rate (23%), while the two artificial observer groups produced lower rates.The third column presents the mean revisit rate for each observer group. Revisit rate was estimated by first creating a null frequency map . Next, for each fixation, the values within a 41 × 41 square (centered at the fixation location) on the frequency map were incremented by 1. This process was repeated for all of the fixations generated by an observer, and the frequency map was then divided by the number of fixations. For each observer, the maximum value from this map was recorded, reflecting the location in the occluded-rod display that was Table 1 indicates, infants tended to have the least-disperse gaze patterns. Fixation dispersion in the image-saliency observer group did not differ significantly from the infant group, although it was significantly higher in the random-observer group . Finally, the fifth column presents the mean gaze shift distance (measured in pixels) for each group. Because this measure was yoked across groups, as expected, the artificial-observer groups produced mean gaze-shift distances that were comparable to the infants’ mean distance.The last two columns present kinematic measures of the gaze patterns. First, dispersion was computed by calculating the centroid of the fixations , then calculating the mean distance of the fixations (in pixels) from the centroid for each observer, and then averaging the resulting dispersion values for each group. As Figure, The final step, prior to training the model, was the process of mapping each set of gaze patterns into a sequence of COG image samples. This was accomplished by determining the frame number that corresponded to the start of each fixation, projecting the gaze point on to the resulting still-frame image, and then sampling a 41 × 41 pixel image, centered at that location. The dimensions of the COG sample were derived from the display size and infants’ viewing distance, and correspond to a visual angle of 1.8°, which falls within the estimated range of the angle subtended by the human fovea . In ordeRecall that our primary hypothesis was that infants’ COG sequences would be more easily learned by an SRN than the sequences from the two artificial-observer models. To evaluate this hypothesis, we trained a set of 3-layer Elman networks, with recurrent connections from the hidden layer back to the input layer , and two input units that encoded the x- and y-coordinates of the upcoming COG sample . The input layer was fully connected to the hidden layer , which in turn was fully connected to the output layer (1681 units). The standard logistic function was used at the hidden and output layers to maintain activation values between 0 and 1; in addition, the bias terms were fixed to 0 for the hidden and output units.An individual training trial proceeded as follows: given the selection of a COG sequence, the first COG sample in the sequence was presented to the SRN. For this first sample, the activation of the context units was set to 0.5. Activity in the network was propagated forward, resulting in the predicted next COG sample. This output was compared to the second COG sample in the sequence, and the root mean-squared error (RMSE) was calculated. Next, the standard backpropogation-of-error  learning algorithm was used to adjust the SRN’s connection weights . The activation values from the hidden layer were then copied back to the input layer, and the second COG sample was presented to the SRN. This process continued until the second-to-last COG sample in the sequence was presented.A total of 10 training runs were conducted. At the start of each run, a single SRN was initialized with random connection weights between 0 and 1, which were then divided by the number of incoming units to the given layer . This network was cloned 12 times, once for each of the infants. This duplication process ensured that any subsequent performance differences between SRNs during a run were due to the training samples unique to each infant, rather than to the initialization procedure.Accordingly, each of the 12 SRNs was paired with one of the infants, and subsequently trained on the three COG sequences associated with that infant: the selected infant’s sequence, as well as the image-saliency and random-gaze sequences that were yoked to the same infant. A single training epoch was defined as a sweep through the three COG sequences. Order of observer type  was randomized for each epoch. Pilot data collection indicated that the SRNs reached asymptotic performance, with a learning rate of 0.1, between 200 and 300 training epochs. As a result, each training run continued for 300 epochs.In order to evaluate our second hypothesis – that resetting the activation of the context layer would have the largest interference effect on the infants’ COG sequences – we “paused” training every 10 epochs to test each of the SRNs. During the testing phase, learning was turned off and all connections were frozen in the SRN. Next, the SRN was tested by presenting the three COG sequences, four times each: (1) with recurrence functioning normally, and (2–4) with the activity of the context units reset to 0.5 every 1, 2, or 5 input steps, respectively.Two sets of planned analyses were conducted. First, we converted RMSE values into rank scores, and then compared the performance of the 12 SRNs as a function of mean rank of each observer group. In particular, this analysis focused on our predictions that the COG sequences from the infant group would have the highest mean ranking at the start of training, and that this difference would persist throughout the training period. The second analysis examined the influence of resetting the context-layer units on the SRNs’ performance, which allowed us to indirectly measure the presence of temporal dependencies in the COG sequences, between both adjacent samples as well as those as many as five samples apart.Figure 4 presents the RMSE produced by the 12 SRNs during the 300 training epochs, as a function of the observer group . Note that these data are pooled over the 12 SRNs and the 10 training runs. In addition, the RMSE values presented in Figure 4 were those generated by the SRNs during the test phase, that is, in which learning was turned off every 10 epochs. As a result, these data reflect the performance of the SRNs while removing the transient effect of testing order .Figure 4. First, the RMSE values produced by the image-saliency group remain consistently highest during training. Second, there is an early “trade-off” between the infant and random-gaze groups, which eventually results in a stable difference, favoring the infant group. In order to determine whether these trends were statistically reliable, we first converted the RMSE values into ranks. In particular, for each epoch, the RMSE for the three observer groups were sorted in ascending order, and assigned the corresponding rank . As before, ranks were then averaged over the 12 SRNs and 10 training runs.There are two important trends suggested by Figure 5 presents the rank-transformed performance data. . In order to compare the three observer groups, a 2-way ANOVA was conducted with epoch and observer group as the two factors. As expected, there was a main effect of observer group . We examined this effect with planned paired comparisons between the three groups (using Bonferroni corrections), which also confirmed our prediction: specifically, the infant observer group had significantly higher overall mean rank than the image-saliency and random-gaze groups. However, these findings were qualified by a significant epoch × observer group interaction . As Figure 5 indicates, near the start of training, the infant and random-gaze groups had similar ranks; in contrast, a large, stable difference between the two groups emerged after approximately 50 epochs.post hoc analysis by first dividing training time into two phases (0 to 50 and 60 to 300 epochs). We then repeated the previous 2-way ANOVA for each phase , including comparisons between the three observer groups. This analysis revealed that while there was no significant difference between the infant and random-gaze groups during the first 50 epochs (p = 0.64), the infant group averaged a significantly higher rank than the random-gaze group during the remaining 250 epochs (p < 0.005). In particular, these results confirm our prediction that the infant observer group would be ranked highest at the start of training, albeit after an initial period of equivalent performance in two of the three groups. In addition, the stability of this pattern for the remainder of the training phase also provides support for our prediction that the infant observer group would maintain the highest rank throughout training.In order to examine this interaction, we conducted a Figure 4), but also under three conditions in which the context layer was reset  after every 1, 2, or 5 training samples. Because it was anticipated that resetting the context layer would produce an increase in prediction errors, RMSE difference scores were therefore computed between each of the reset conditions and the canonical condition. These difference scores were then transformed into percent-change scores, relative to the canonical condition . Figure 6 presents the resulting percent-change values for each of the observer groups, within the three reset conditions .The second set of analyses focused on the role of the context layer in the SRN architecture, and more specifically, on the question of whether periodically resetting the activity of this layer during training would disrupt performance. In order to address this question, recall that during each test phase, each of the SRNs was not only tested under canonical conditions  > 3.87, ps < 0.001]. In general, as Figure 6 illustrates, this interaction reflects the tendency for percent-change scores to begin near 0 for each of the observer groups, and then subsequently increase at different rates over training time. We pursued this interaction by dividing training time into three blocks of epochs , and then conducting a simple-effects test of observer group for each of the three blocks. Two consistent findings emerged from this test. First, across each of the three training blocks and two of the three reset conditions, the percent increase of the RMSE in the infant group was significantly higher than in the random-gaze group . The only exception to this result was in the condition where the context layer was reset every five samples, during the final block of epochs; in this case, the infant and random-gaze groups did not significantly differ. Second, a significant difference between the infant and saliency groups was not present during the first two blocks of epochs . However, by the third block of epochs, the percent increase in RMSE in the infant group was significantly higher than in the saliency group, for all three reset conditions . Taken together, these findings collectively support our prediction that resetting the context-layer activation values would have the largest interference effect on the infants’ COG sequences.Third, we conducted a 2-way ANOVA for each of the reset conditions, again with epoch and observer groups as the two factors. This comparison revealed a significant epoch × observer group interaction for all three reset conditions [all The current simulation study focused on two goals. First, we sought to demonstrate that our previous gaze-sequence learnability findings, from an infant free-viewing task , would gThe results were consistent with each of our four hypotheses. First, we predicted that infants’ COG sequences would be learned first by the 12 SRNs. We assessed this prediction by converting each observer group’s error scores into ranks and then analyzing the respective ranks over 300 epochs of training time. As we predicted, the infant group eventually established a significant advantage over the other two observer groups. Unexpectedly, however, this advantage did not appear at the onset of training. Instead, the average ranks of the infant and random-gaze groups were comparable for the first 50 epochs of training. One potential explanation for this early similarity of performance in the two observer groups is that there was a higher initial “learning cost” associated with the infant group, due to the (presumed) presence of temporal dependencies in their COG sequences, which ostensibly required additional time for the SRNs to detect and exploit (through the context layer). Second, we also predicted that this advantage would persist and remain stable across the remaining time. Again, the results supported our prediction.a priori regularities or dependencies within the random-gaze model’s COG sequences, other than those broadly present in the display itself .Our third and fourth predictions focused on whether the success of the SRN architecture in learning the infants’ COG sequences benefited from the (presumed) presence of temporal or sequential dependencies embedded within the infants’ COG training samples. Luckily, the use of the random-gaze model provides a critical role in addressing this question, as the gaze sequences from this model were specifically produced with a stochastic procedure . As a result, we can thus assume that there were no Figure 6 illustrates, this prediction was confirmed as well.We therefore predicted that disrupting information flow within the recurrent pathway of the network by periodically resetting the context layer would increase the overall errors produced by the SRNs. Indeed, across all three observer groups we observed significant increases in the SRN prediction errors when the recurrent layer was reset. Our last prediction was that the interference effect would be greatest for the infants’ COG sequences, and as Figure 6 may offer three additional insights. First, as we suggested above, the gaze sequences produced by the random-gaze model should include minimal (if any) sequential structure. Nevertheless, note that – like the other two observer groups – the interference effect increased with training time in the random-gaze group. This trend provides a statistical baseline for estimating the contribution of the context layer for prediction learning on the current task, as the training sequences from the random-gaze model were ostensibly sequentially independent. We can therefore estimate the presence of any additional structure embedded within infants’ COG sequences by subtracting the RMSE change values produced by the random-gaze model. For example, in the first reset condition  and pooling over training time, the overall difference in RMSE change between the infant and random-gaze groups is 42%. This value provides an important clue toward understanding the function of infants’ object-directed gaze behavior, as it demonstrates that infants’ gaze sequences are significantly more structured than sequences produced by chance, and that this embedded sequential structure also provides a measurable advantage to an active observer that is learning to forecast or predict the content of upcoming fixations.Further inspection of An additional insight offered by manipulating the context layer is reflected by the regular order of performance observed across the three observer groups. In particular, note that the interference effect was consistently lowest in the random-gaze group, highest in the infant group, and midway between the two in the image-saliency group. This finding suggests that the simple strategy of orienting toward relatively high-saliency regions in the occluded-rod display is sufficient to generate statistically reliable temporal structure in the COG sequences.Table 1 indicates, the infant and image-saliency groups fixated regions of the occluded-rod display that were on average nearly equal in salience. Second, as Figure 6 illustrates, resetting the context layer had a comparable effect on the infant and image-saliency groups during the first 75–80 epochs of training .Finally, a third insight suggested by these findings is that image-saliency may provide, at best, a partial account for how infants’ gaze patterns are structured over time and space. In particular, our previous work has demonstrated that a saliency-based model captures several global-level features of infants’ gaze patterns, such as the frequency of fixations toward the rod segments, as well as individual differences in the rate of rod fixations between infants , 2012. Ithe particular temporal order in which infants scanned salient regions of the display that provided an additional predictive cue to the SRNs. We are currently exploring computational strategies for teasing apart these spatial and temporal cues, and isolating their influence on the prediction-learning process.However, after approximately 80 epochs, the interference effect continued to increase at a faster rate in the infant group. One potential interpretation for this pattern is that, due to similar levels of saliency in the infants’ and image-saliency models’ COG samples, the SRNs “focused” during early learning on saliency-related features in the input  as a predictive cue. In contrast, the random-gaze model fixated salient locations less frequently , and as a result, recurrent feedback in the SRN had less impact on prediction learning for the sequences from this observer model. If this reasoning is correct, it suggests that the subsequent performance split between the infant and image-saliency models was presumably due to additional temporal structure – beyond that provided by saliency – in the infants’ sequences, which the SRNs continued to learn to detect and exploit. To put the point concisely: while infants and the image-saliency model fixated (on average) equally-salient regions in the occluded-rod display, we are proposing that it was Two key issues remain unaddressed by our work thus far. First, it is important to note that our use of the SRN architecture, as well as our manipulation of the context layer, provide a somewhat indirect method for identifying sequential structure in infants’ COG samples. In general, this strategy tells us that temporal structure is present and it also provides a method for quantifying the interference caused by periodically resetting the context units, but it does not directly identify the visual features detected by the SRN, not does it specify how variation in these cues over time  improves the outcome of sequence learning. An additional limitation of the reset method, which we noted in the introduction, is that the samples that are processed before a reset occurs do not contribute equally to the memory trace that accumulates in the recurrent pathway .There are several strategies available to address these issues. For example, alternative analytical methods  as well as alternative modeling architectures and learning algorithms  may provide additional insights. We are also currently exploring the strategy of constructing artificial gaze sequences in which we strictly control the statistical dependencies over time . Ideally, this will allow us to examine the influence of resetting the context layer versus learning/detecting temporal dependencies that vary in their duration over time. A related limitation of the modeling strategy we have employed here is that the SRNs were trained over multiple repetitions of the same COG sequences. In particular, this repetition provides an important learning cue to the SRNs, independent of the temporal structure embedded within the COG sequences. One way to address this issue is to employ a “leave-out” training regime, in which a subset of training patterns are set aside and reserved for testing the model.their COG sequences will become more predictable (due to greater similarity between sequences). In addition, recall that after habituating to the occluded-rod display, infants then view the solid-rod and broken-rod test displays . Therefore, a related question is whether predictability of the COG sequences will increase or decrease during the test trials, and in particular, whether it will vary across the two display types. Answering these questions is essential to understanding the role of visual prediction-learning during the development of object perception.Second, we should also note that our current simulation study focused exclusively on infants’ first trial during the perceptual-completion task. An open question is whether infants’ scanning patterns change systematically over subsequent trials , and if so, what effect if any will such changes have on the predictability of the COG sequences that are produced during later trials? Our intuition is that if infants’ gaze patterns during later trials are less variable , We now return to the issue of early object-perception development in young infants. Our work has not only implicated the role of active visual scanning as an essential skill for object perception , but alsafter they have been produced. Thus, one of our long-term goals is to design an architecture that can accurately forecast infants’ upcoming fixations before they are produced. One application of this forecasting technique would then be to manipulate the features or properties of the gaze destination before the infant gazed at that location, as a way of gauging their sensitivity to those features . Second, we have previously observed variation across infants at the same age with visual displays such as the perceptual-completion task (e.g., Finally, we conclude by noting that our modeling approach has the potential to offer two important innovations for the study of perceptual development in infants. First, our current strategy is to analyze infants’ COG sequences offline, that is, sk e.g., . We are The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The alignment of multiple protein sequences is one of the most commonly performed tasks in bioinformatics. In spite of considerable research and efforts that have been recently deployed for improving the performance of multiple sequence alignment (MSA) algorithms, finding a highly accurate alignment between multiple protein sequences is still a challenging problem.We propose a novel and efficient algorithm called, MSAIndelFR, for multiple sequence alignment using the information on the predicted locations of IndelFRs and the computed average log–loss values obtained from IndelFR predictors, each of which is designed for a different protein fold. We demonstrate that the introduction of a new variable gap penalty function based on the predicted locations of the IndelFRs and the computed average log–loss values into the proposed algorithm substantially improves the protein alignment accuracy. This is illustrated by evaluating the performance of the algorithm in aligning sequences belonging to the protein folds for which the IndelFR predictors already exist and by using the reference alignments of the four popular benchmarks, BAliBASE 3.0, OXBENCH, PREFAB 4.0, and SABRE (SABmark 1.65).sum–of–pairs and total column metrics.We have proposed a novel and efficient algorithm, the MSAIndelFR algorithm, for multiple protein sequence alignment incorporating a new variable gap penalty function. It is shown that the performance of the proposed algorithm is superior to that of the most–widely used alignment algorithms, Clustal W2, Clustal Omega, Kalign2, MSAProbs, MAFFT, MUSCLE, ProbCons and Probalign, in terms of both the The online version of this article (doi:10.1186/s12859-015-0826-3) contains supplementary material, which is available to authorized users. Alignment of multiple protein sequences is a crucial step in bioinformatics analyses, and is used in many applications including sequence annotation, phylogenetic tree estimation, evolutionary analysis, secondary structure prediction and protein database search , 2. Multprogressive alignment algorithms, while MAFFT, MUSCLE, ProbCons and Probalign generate an initial alignment using the progressive alignment algorithm and then iteratively refine this alignment to achieve higher alignment accuracy. A progressive alignment algorithm involves three steps: (i) calculations of the pairwise distances between all pairs of sequences to determine the similarity of each pair of sequences, (ii) construction of a guide tree based on the distance matrix, and (iii) finally, alignment of the sequences according to an order determined by the guide tree =a1a2⋯ai be a prefix of sequence A, B [ 1:j]=b1b2⋯bj be a prefix of sequence B, M be the optimal score for aligning A [ 1:i] and B [ 1:j] given that ai is aligned to bj, IA be the optimal score given that ai is aligned to a gap, and IB be the optimal score given that bj is aligned to a gap, where 1≤i≤n and 1≤j≤m. The recursive equations to find the various elements in the state matrices M, IA, and IB are given byLet withs can be obtained directly from the substitution matrix S, A, and B. Once the computation of M is completed, it contains the maximum alignment score, and a trace back procedure is used to retrieve the alignment between A and B.where n and m (n≥m), with a time complexity of O(mn) and a space complexity of O(n). Since it has been shown in [In this article, we implement the memory efficient DP algorithm proposed in , which cshown in  that theshown in , PAM [35shown in  as the sGPOi and GPEi, for each profile. For example, consider the alignment of two sequences, say, A and B at the lowest level of the tree to produce the profile C. The position–specific gap opening penalty function for profile C is defined to beIn order to continue aligning sequences/profiles until the highest level of the guide tree is reached, we need the gap penalty functions: j, k, and i for A, B and C, respectively. In a similar manner, we define the gap extension penalty function for C. This makes a gap more likely to occur at a position, where a gap already exists. If there is no gap at a position i in C, then the gap opening penalty is increased by adding both where As already mentioned, the internal nodes of the guide tree are visited in a bottom–up order, and for each visited node a pairwise alignment of sequences/profiles is computed using the DP approach along with the proposed VGP function. The MSA associated with the root node is the final alignment.auto option is used with the maximum iterative refinement (maxiterate option) set to 1000, while the default options are used for all the other algorithms, including the proposed MSAIndelFR.The performance of MSA algorithms are usually evaluated on alignment benchmarks containing reference alignments. In this article, we use four popular benchmarks, namely, BAliBASE 3.0 , OXBENCHIn the present article, we select the reference alignments from the above four benchmarks that have protein sequences belonging to one of the 43 protein folds  and total columns (TC) [core blocks of the reference alignment, where the core blocks of the reference alignment refer to the regions for which reliable alignments are known to exist. We use the BENCH database  to determine the core blocks in the selected benchmarks. It should be noted that the quality(Q) metric used in [1 In order to determine if the improvements, achieved in terms of the SP and TC values, by the proposed MSAIndelFR algorithm are statistically significant, the Wilcoxon matched-pair signed-rank test [We use the measures, mns (TC) , which a used in  is equiv used in .1 In ordank test  is used.For evaluating multiple sequence alignment algorithms, BAliBASE  is the m% representing an improvement of 6.02 %, 1.37 %, 4.12 %, 4.29 %, 6.17 %, 10.37 %, 0.39 % and 0.78 % over that of MSAProbs, MAFFT, MUSCLE, Clustal Omega, Kalign2, Clustal W2, ProbCons and Probalign respectively. Also, it provides an average TC value of 57.56 % representing an improvement of 2.62 %, 3.06 %, 10.15 %, 7.20 %, 13.87 %, 18.19 %, 2.74 % and 3.92 %, respectively, over that of the other alignment algorithms.The average SP and TC values of MSAIndelFR as well as those of the other algorithms using this benchmark as reference are given in Table Boxplots would show more detailed information about the distribution of the SP and TC values than that provided by Table %, the distribution of the SP values generated by MSAIndelFR is much narrower than that generated by MSAProbs, since the former has an IQR value of 12 %, whereas the latter a value of 20 %. In addition, it is seen that 75 % of the MSAIndelFR alignments have an SP value of more than 84 % (first quartile), whereas 25 % of the alignments have an SP value of more than 96 % (third quartile). Figure In view of the above reasons, boxplots resulting from the distributions of the SP values of the various algorithms evaluated using BAliBASE 3.0 are shown in Fig. The OXBENCH benchmark  is a set% representing an improvement of 2.49 %, 3.65 %, 3.22 %, 3.83 %, 4.33 %, 3.94 %, 3.02 % and 2.80 % over that of MSAProbs, MAFFT, MUSCLE, Clustal Omega, Kalign2, Clustal W2, ProbCons and Probalign, respectively. Also, it provides an average TC value of 83.83 % representing an improvement of 4.05 %, 5.85 %, 4.90 %, 6.07 %, 6.53 %, 6.83 %, 5.02 % and 4.31 %, respectively, over that of the other alignment algorithms.The average SP and TC values of MSAIndelFR as well as those of the other algorithms using this benchmark as reference are given in Table % of the MSAIndelFR alignments have an SP value of more than 91 % (first quartile), whereas 25 % of the alignments have an SP value of 100 % (third quartile).The boxplots for the SP and TC value distributions of the various algorithms are given in Additional file The PREFAB 4.0 benchmark  is a ful% representing an improvement of 1.83 %, 5.42 %, 3.61 %, 3.39 %, 3.02 %, 3.30 %, 2.92 % and 2.72 % over that of MSAProbs, MAFFT, MUSCLE, Clustal Omega, Kalign2, Clustal W2, ProbCons and Probalign, respectively. Also, it provides a similar TC improvements over the other algorithms.The average SP and TC values of MSAIndelFR as well as those of the other algorithms using this benchmark as reference are given in Table % of the MSAIndelFR alignments have an SP value of more than 31 % (first quartile), whereas 25 % of the alignments have an SP value of 88 % (third quartile).The boxplots for the SP and TC value distributions of the various algorithms are given in Additional file % in the Superfamily set, while it is at most 25 % in the Twilight set. In [The SABmark 1.65  is a ver set. In , the aut set. In , contain% representing an improvement of 2.04 %, 3.45 %, 7.25 %, 8.48 %, 11.94 %, 13.21 %, 2.32 % and 3.25 % over that of MSAProbs, MAFFT, MUSCLE, Clustal Omega, Kalign2, Clustal W2, ProbCons and Probalign, respectively. Also, it provides an average TC value of 34.38 % representing an improvement of 9.18 %, 10.06 %, 13.58 %, 14.80 %, 15.48 %, 18.40 %, 9.42 % and 10.71 %, respectively, over that of the other alignment algorithms.The average SP and TC values of MSAIndelFR as well as those of the other algorithms using this benchmark as reference are given in Table %). In addition, it is seen that 75 % of MSAIndelFR alignments have an SP value of more than 29 % (first quartile), whereas 25 % of the alignments have an SP value of more than 77 % (third quartile).The boxplots for the SP and TC value distributions of the various algorithms are given in Additional file p-values obtained by the Wilcoxon matched-pair signed-rank test between the proposed MSAIndelFR and other alignment algorithms for the four benchmarks using the SP and TC scores, respectively. A p-value less than 0.05 is considered to be statistically significant [The Wilcoxon matched-pair signed-rank test  is now unificant , 12, 13.We now compare the run times of the proposed MSAIndelFR and other alignment algorithms using a desktop PC with Intel(R) Core(TM) i7–2600 CPU at 3.40GHZ and RAM of 16GB. As explained earlier, MSAIndelFR needs the computed average log–loss values and the predicted locations of IndelFRs to set the gap penalty functions for each protein sequence in the selected reference alignments from the four benchmarks . In this selection of these sequences, it is ensured that they belong to one of the 43 protein folds for which IndelFR predictors are available. The results have shown that the performance of the proposed MSAIndelFR algorithm is superior to that of the eight most–widely used alignment algorithms, Clustal W2, Clustal Omega, MSAProbs, Kalign2, MAFFT, MUSCLE, ProbCons and Probalign, in terms of both the SP and TC metrics which have been calculated using reference alignments of the four benchmarks. Furthermore, it has been shown that the improvements achieved over all the other algorithms by the proposed algorithm are, in general, statistically significant. It is to be made clear that the concepts behind the proposed alignment algorithm are not restricted to the 43 protein folds considered in this article. These protein folds have been used to illustrate the proposed algorithm. However, if a protein sequence to be aligned belongs to some other protein fold, a new predictor needs to be first constructed and then used in the proposed alignment scheme.The source code is available on request from the authors.1 An example of calculating the SP and TC values is given in Section 2 of the Additional file Additional file 1Supplementary materials. Additional file


This favors the placement of insertions between conserved blocks, which can be understood as making up the proteins’ structural core. (iii) It uses a Bayesian statistical measure of alignment quality based on the minimum description length principle and on Dirichlet mixture priors. Consequently, GISMO aligns sequence regions only when statistically justified. This is unlike methods based on the ad hoc, but widely used, sum-of-the-pairs scoring system, which will align random sequences. (iv) It defines a system for exploring alignment space that provides natural avenues for further experimentation through the development of new sampling strategies for more efficiently escaping from suboptimal traps. GISMO’s superior performance is illustrated using 408 protein sets containing, on average, 235 sequences. These sets correspond to NCBI Conserved Domain Database alignments, which have been manually curated in the light of available crystal structures, and thus provide a means to assess alignment accuracy. GISMO fills a different niche than other MSA programs, namely identifying and aligning a conserved domain present within a large, diverse set of full length sequences. The GISMO program is available at http://gismo.igs.umaryland.edu/.We describe a Bayesian Markov chain Monte Carlo (MCMC) sampler for protein multiple sequence alignment (MSA) that, as implemented in the program GISMO and applied to large numbers of diverse sequences, is more accurate than the popular MSA programs MUSCLE, MAFFT, Clustal-Ω and Kalign. Features of GISMO central to its performance are:  ad hoc measures of alignment quality, and (iii) pre-specified, uniformly-defined gap penalties. Here we describe an alternative strategy that first provisionally aligns regions generally shared by all the input sequences, and then refines this alignment by iteratively realigning correlated sequences in tandem. It infers position-specific gap penalties directly from the evolving alignment. It avoids suboptimal traps by stochastically traversing the complex, correlated space of alignments using a statistically rigorous measure of alignment quality. For large sequence sets, this approach offers clear advantages in alignment accuracy over the most popular programs currently available.Existing multiple alignment programs typically utilize (i) bottom-up progressive strategies, which require the time-consuming alignment of each pair of input sequences, (ii)  PLOS Computational Biology Methods paper.This is a A common starting point for the computational analysis of proteins is the construction of a multiple sequence alignment (MSA). Insofar as they result from protein functional similarities and differences, the patterns of residue conservation and divergence within such an alignment provide clues to biological function. Of course the biological relevance of any observed patterns depends upon an alignment’s accuracy, and alignments of larger sequence sets have greater statistical power. For biologically appropriate scoring systems applied to more than a very small number of sequences, however, no optimization procedures are known that are both tractable and rigorous; thus all practical MSA programs rely upon heuristic methods.n sequences of a fixed average length, it requires O(n2) time to compute such scores, and this becomes the time-dominating step for large n. One way around this problem is to iteratively refine a guide tree and MSA starting from an initial crude guide tree, an approach used in the popular MUSCLE  ≡ Aj − {ak,j} to denote the set Aj without ak,j. An alignment is defined by the matrix A = T and {A} ≡ {ak,j: k = 1,..,K, j = 1,…,w} denotes the set of residues indices for the alignment variable A. We represent the collection of residues indexed by elements in a set C as RC. For instance, RA}{ = {ak,j: k = 1,…,K; j = 1,…w} represents the set of residues in the alignment defined by A.A block-based alignment of the input sequences is defined by c are modeled as a multinomial distribution with parameters θc = T where θi,c > 0 for all i. That is, the vector Θ =  defines a product multinomial model corresponding to the full alignment. The vector θ0 corresponds to a background amino acid residue distribution. Hence, the complete-data likelihood function is given byD(B) and θ0 ∼ D(α) (where D denotes the Dirichlet distribution), and where B =  specifies the Dirichlet distribution parameters (commonly interpreted as numbers of pseudocounts) at each column position j, and α specifies the parameters for the background distribution.  = K×w where ak,j indicates the position of the j-th column, which is assumed to be present in all of the sequences.) The likelihood of A with the θ’s integrated out isThe residue frequencies observed for column i in sequence k is given byθ, given the observed sequence data R and the current alignment Ak][−. This statistical model serves as the foundation for the HMM [The conditional predictive probability distribution of this conserved region occurring at position  the HMM  used in In order to capture the fact that certain biochemically or structurally similar amino acid residues are more likely to occur together we have incorporated Dirichlet Mixture priors , 16, as k a non-integer weight is computed using the method of Henikoff and Henikoff [j and where j as for sequence k. The rationale for this formulation is that if a sequence matches lots of sequences at most positions, then it should receive a lower weight than a sequence that matches few sequences at most positions. These weights are then normalized and integerized as:wtmax corresponds to the maximum non-integer sequence weight. Because these weights depend upon the evolving alignment, they are updated after each sampling cycle.Sequences are down weighted for redundancy using the following procedure. For each sequence Henikoff  as:wt estimates for the transition probabilities at each position j are computed from these observed and prior counts. These define the position specific gap penalties. The joint posterior distribution for the alignment and transition probability parameters isP is a generalization of Equation (1), and where w vectors representing the column-specific transition probabilities with prior probability:Ignoring the indexing variable Given the alignment and thus the paths Λ, we have the conditional posterior distributionj is done by drawing the random variables:Sampling on the distribution for each position For computational efficiency, the ι and δ may be integrated out  to gethg ∝ P × h(Λ), for which the transition probability parameters need not be fixed or updated and which allows the optimal indel penalties to be determined from the sequence data.This gives rise to a new posterior distribution T = 1  and ends at T = 0 ; between these two extremes the temperature is dropped in ΔT = 0.1 increments with sampling probabilities set to GISMO’s MCMC sampling algorithm explores the space of possible alignments by executing Markovian transitions between alignments. This involves sampling alternative alignments of either individual sequences or groups of sequences. In either case, such sampling is done as follows: First, the sequence or sequences are removed from the alignment and the posterior parameters of the HMM are recalculated based on the retained aligned sequences and the priors. Next, emission probabilities for the twenty amino acids at each position are sampled from the posterior emission probability distributions defined by the HMM parameters; note that these sampled probabilities define a sampled HMM. Finally, the previously removed sequences are optimally realigned to the sampled HMM. We explored sampling transition probabilities in the same way, but found little benefit of doing so; instead, the MAP estimates for transition probabilities are used. GISMO applies simulated annealing  to favorhttp://gismo.igs.umaryland.edu/.The GISMO program and the CDD benchmark MSAs and sequence sets used for this study are available at S1 Statistics(XLSX)Click here for additional data file.S2 Statistics(XLSX)Click here for additional data file.S1 Text(PDF)Click here for additional data file.S1 Charts(XLSX)Click here for additional data file.S2 Charts(XLSX)Click here for additional data file.S3 Charts(XLSX)Click here for additional data file.S1 Runtimes(XLSX)Click here for additional data file.S1 FigThe nine sequences between the two lines are misaligned; the insert residues shown in red correspond structurally to the first 10 columns shown. Note that these misaligned sequences share two distinguishing features: (i) they contain 27–30 residue insertions that the other sequences lack and they conserve a glycine (G) residue in the seventh column instead of the consensus arginine (R) residue. GISMO relies on such features to identify and realign clusters of correlated sequences.(PDF)Click here for additional data file.S2 FigThis corresponds to the same sequences and domain footprint as the MAFFT alignment in (PDF)Click here for additional data file.S3 FigThis corresponds to the same sequences and domain footprint as the GISMO alignment in (PDF)Click here for additional data file.S4 FigThis corresponds to the same sequences and domain footprint as the MAFFT alignment in (PDF)Click here for additional data file.S5 FigThis corresponds to the same sequences and domain footprint as the GISMO alignment in (PDF)Click here for additional data file.S6 FigThis corresponds to the same sequences and domain footprint as the MAFFT alignment in (PDF)Click here for additional data file.S7 FigThis corresponds to the same sequences and domain footprint as the GISMO alignment in (PDF)Click here for additional data file.S8 Fig(PDF)Click here for additional data file.


Measuring sequence similarity is central for many problems in bioinformatics. In several contexts alignment-free techniques based on exact occurrences of substrings are faster, but also less accurate, than alignment-based approaches. Recently, several studies attempted to bridge the accuracy gap with the introduction of approximate matches in the definition of composition-based similarity measures.MissMax, an exact algorithm for the computation of the longest common substring with mismatches between each suffix of a sequence x and a sequence y. This collection of statistics is useful for the computation of two similarity measures: the longest and the average common substring with k mismatches. As a further contribution we provide a “relaxed” version of MissMax that does not guarantee the exact solution, but it is faster in practice and still very precise.In this work we present  Sequence similarity has long been playing a crucial role in Computational Biology and Bioinformatics as a key ingredient in the prediction of functional and structural properties, and of evolutionary mechanisms.Since the introduction of high throughput techniques, hundreds of fully sequenced genomes of different species have been made available at a fast pace. The increasing number of available sequences makes all kind of sequence analysis, most notably assembly, phylogenetic reconstruction, and multiple alignments, more challenging due to the time consuming and memory-demanding operations that need to be carried out on these huge datasets.To try to cope with the increasing demand of time efficiency, a wide range of alignment-free (or composition-based) approaches have been proposed. The idea behind compositional approaches is to model each sequence in terms of the substrings that it contains, and then to devise appropriate similarity measures to compare two sequences based on this model .L-mers, for a fixed length L, and consider exact matches. Although usually very fast, in several contexts such approaches can be much less accurate than alignment-based counter-parts.Traditionally, alignment-free approaches rely on the frequency or presence of For this reason, within the last decade, several approaches have been proposed to improve the ability to better capture the nature of the similarity/dissimilarity between biological sequences with alignment-free techniques. Among the wide literature, we can mention, for example, the introduction of over-representation, rather than the raw frequency count, in the definition of the similarity measure for fixed length  and maxiMore recently, several studies proposed to model the intrinsic variability of biosequences by considering approximate matches with a bounded number of mismatches, or by using spaced-words, in the characterization of the sequence composition. Several related experiments showed that, in the context of phylogenetic tree reconstruction, the introduction of approximate matches can improve the quality of the detected sequence similarity , 13, 14.Given these premises, we focused our attention on these more involved formulations of the alignment-free approach, in particular on those allowing for approximate matching within a bounded number of mismatches.As a warm up we will give a brief overview of recently proposed approaches, and will use some of the presented results to introduce further notation used throughout the paper.x and y starting at position i and j respectively. In the following we will assume, without loss of generality, that both sequences have the same length n.Let us consider two sequences k mismatches of all the substrings of length L in a string x of length n. The key feature of this algorithm is that its complexity is independent on the number of mismatches that are allowed. The algorithm was proposed within the pattern discovery framework [L of a string x in another string y, leading to the definition of a similarity measure between the two sequences based on the frequency of shared approximate occurrences.An early result on speed-ups for the computation of empirical statistic with mismatches was presented in , where aramework , 8, thusmaximal substrings with mismatches was introduced in [x and y when k mismatches are allowed. Now, consider the set of x, and for all the suffixes y.The first formal definition of similarity measures based on shared duced in . We repok:The following measures of cross correlation were defined for a given number of mismatches n-length vector storing the maximum value attained by i over all values correspondingly spanned by j.n-length vector storing the average value attained by i over all values correspondingly spanned by j.x and y.x and y.For measures such as vCork in  it was akmacs, a greedy heuristic, was proposed to generalize the well known Average Common Substring (ACS) distance [k mismatches when considering the longest common substring between pairs of positions in the two strings. We refer to this variant of the ACS problem as kACS. The algorithm proposed in [O(nkz), where z is the maximum number of occurrences in y of a string of maximal length occurring in both x and y. Being based on a heuristic, this method is very fast in practice, but it does not guarantee to find the optimal solution to the problem. Note that the kACS problem can be described in terms of the measures of cross correlation previously defined as the mean over all positions i in x of In  kmacs, adistance  so to acposed in  has timek-LCF problem is introduced as the generalization of the longest common substring (there named “factor” to avoid confusion with LCS as the Longest Common Subsequence problem) as finding the longest common shared match between two sequences when up to k mismatches are allowed. Also this problem can be described in terms of the previously defined scheme, as it corresponds to O(nm) time and O(1) space solution is provided for a generic k and two strings of length n and m respectively, and also an O(n) space algorithm was proposed to provide a subquadratic solution to the kACS problem.We end our overview with some recently published theoretical results. In  the k-LCentclass1pt{minimax and another sequence y. This primitive is at the basis of recently proposed similarity measures for the problem of phylogentic tree reconstruction as the longest and the average common substring distances with mismatches.Within this framework we developed a new strategy based on filtering for the computation of the longest common substring with mismatches between all the suffixes of a sequence k-LCF) and of kACS, being respectively:Formally, we will compute the values of i in x. The main idea behind the proposed approach is to avoid the computation of the Our aim is to compute the values of i, one can either take track of their maximum value to compute Note that, when computing the In the description of the algorithm we will refer to techniques that allow us to keep the worst case analysis within the claimed worst case quadratic bound, but for practical purposes we will use different approaches that will be discussed in a dedicated subsection.k mismatches of the suffix y. For this purpose we will start from, and exploit, the classical concept of longest common substring .We start with the computation of x and y, of length n, find the maximum length L for which a pair of indexes  exists such that Given two sequences i in x and j in y. This problem can be solved through a call to the Lowest Common Ancestor of the corresponding leaves i and j in constant time after a linear-time preprocessing step [The problem of finding the longest common substring between two sequences is a well known problem in pattern matching that can be solved in linear time by the traversal of a generalized suffix tree of the two sequences. More in details, we want to be able to find the longest common match starting at any two positions ing step .jump-extensions to compute the longest approximate match between O(kn) time overall.In particular, similarly to the routine step in , we willj be the position in y of a longest approximate match of k mismatches all lie within y.L positions we know we must have a mismatch, which is now counted as the k-th. To finally obtain y.Assume now we have computed mentclass2pt{minimy are the site of a longest match with k mismatches with k mismatches from the previous step at the minimum cost (only a jump till the next mismatch to the right is needed to have the exact length for the pairs of positions considered at this step);to obtain the longest possible candidate length with to avoid or reduce the possibility that we have to deal with special cases in the following steps  candidate pairs per position i, and to a worst case time complexity Observation 2. We observe that even in the worst case, we can rule out all the positions y that are preceded by a character that matches y.Observation 3. Whenever we impose a matching condition we potentially reduce the candidate set of a fraction equal to Observation 4. At step i is k-th mismatch rather than on the first position after that, thus ending up to erroneously discard position Observation 5. The case discussed in Observation 4 is very time consuming because the candidate set remains pretty large. However, we have verified in practice that the number of ties that are actually found during that step is very small, and orders of magnitude less than the size of Observation 6. Building indexing data structures can be expensive, and so can be operations that are theoretically efficient. For example, it was already observed in [k mismatches gave better performances than calling LCA  based distance discussed in [In this section we present the results of a set of experiments that we run to test the performances of the exact and relaxed version of MissMax. Here we are not mainly interested in the improvement of the quality of the tree reconstruction with mismatches, with respect of corresponding measures without mismatches, as they were already discussed in , 14. We ussed in , and then values of For what concerns the comparison with other algorithms, the algorithm described in , which hof kmacs , is uncoFor our experiments we considered two datasets that were previously used in other studies e.g. , 14, 18), 1814, 1k, the average time for the comparison of a pair of sequences in each dataset for the relaxed and the exact version of MissMax, respectively.As a first experiment we measured the time performances of both the exact and the “relaxed” version of MissMax on omogeneous and heterogeneous subsets of the 34 mammals datasets. In particular, we considered the subset of Rodents , the subset of Carnivora , and a mixed set of the two. We then measured the average time needed to compute the similarity between two sequences within each of the omogeneous set, and between elements of different sets in the mixed subset. Finally we added to our analysis a set of 5 random sequences and measured the performances within the subset. Tables It is immediate to observe that the relaxed version of MissMax takes about half of the time needed by the exact version. Nevertheless, even the exact algorithm allows for a full pairwise comparison of typical datasets in about an hour on our desktop computer. The time required for each comparison is consistent with the one showed for the smaller datasets, thus the time required for the overall analysis is a function of the size of the dataset. For example, with k increases.With respect to the kind of sequences that are analized, we can see how both algorithms are faster on the random dataset than on the biological datasets. The difference is much more evident for the exact version of the algorithm, when kmacs is generally much faster than MissMax: on such small datasets it ends practically istantaneously. We recall that the theoretical time complexity of kmac is O(kzn), where z is the number of positions in which there is a tie for the first longest exact match. It is shown in [z are usually pretty small. Nevertheless, we report, for completeness of discussion, the following observation. While running our experiments, we had a case in which, increasing the length of the sequences to compare, kmacs performances had a suddent drop. After ruling out this could depend on the actual input length of the sequences (the performances were good on random sequences of the same length), or on some bug (the results were what we were expecting), a closer inspection of the input sequences revealed the probable cause. More or less in correspondance of the input length that was showing the slow down, there was an undefined region in one of the two sequences. Note that the computed value of the longest matches of suffixes was not affected, as the N region was present just in one sequence. Nevertheless, the first exact longest match of a suffix starting in the N region is 0, and in such a case, z becomes equal to n, and all the positions of the other sequence need to be extended to find the maximum extension with mismatches.For comparison with other approaches, we run the naive algorithm on the Rodents datasets. The average time was: 7.1 s for shown in  that in k, on sequences up to 100k bp, as a function of the input length. The trends for the exact version are the same. We reported a slowdown of about 2, with the actual values slightly increasing with k. Specifically, the average slowdown factor was 1.87 for Finally, to further investigate the scalability of the filter-based approach we performed a similar test on longer sequences. The results are reported in Fig. The tradeoff between exact algorithms and heuristics could be easily summarized saying that heuristics are faster, but do not guarantee the correct solution to the problem. This is of course true also in our case, but we performed anyway some further analysis to assess the performances of MissMax (both exact and relaxed version) and kmacs, in terms of precision in the computation of sequence similarity based on the aforementioned distances with mismatches.k’s have been tested on the same dataset, but the exact reference phylogeny was never captured).In terms of reconstruction of the correct phylogeny, we tested the algorithms on the Primates datasets. MissMax reported the same reference tree as in  (see Fignces (in  a wide rAs previosuly stated, kmacs is usually much faster even than the relaxed version of MissMax. However, in terms of computation of the actual value of the average common substring between two sequences, relaxed MissMax is much more precise. In fact, we measured a relative error of 0.53 % with respect to the real measure for both the Rodents and the Carnivora datasets, and 0.46 % for the mixed dataset. For the full Primate dateset, we measured the relative error for both orted in , this doA set of experiments was performed to measure the goodness of our filters, investigating on the percentage of pair of positions that are actually considered with respect to the maximum possible. The results are shown in Table k up to We note that in the biological sequences the difference between the size of the candidate set of the relaxed and exact version is quite small for It is interesting to note that the analysis on random sequences is faster than in biological sequences (see Tables k mismatches between each suffix of a sequence x and a sequence y we want to compare to. This statistics is useful for the computation of alignment free distances based on approximate matching, that are a promising approach to improve the quality of alignment free sequence comparison. We developed both an exact and a relaxed version of the algorithm. While the relaxed version cannot guarantee to find the optimal solution, it is in practice faster and still very precise.In this work we proposed a filtering-based approach for the computation of the longest common substring with


All computations are performed at server side. Interactivity is implemented in HTML5, a language native to web browsers. The alignment is initially displayed using default settings and can be modified with the graphical user interfaces. For example, individual sequences can be reordered or deleted using drag and drop, amino acid color code schemes can be applied and annotations can be added. Annotations can be made manually or imported . Some edits take immediate effect while others require server interaction and may take a few seconds to execute. The final alignment document can be downloaded as a zip-archive containing the HTML files. Because of the use of HTML the resulting interactive alignment can be viewed on any platform including Windows, Mac OS X, Linux, Android and iOS in any standard web browser. Importantly, no plugins nor Java are required and therefore Alignment-Anotator represents the first interactive browser-based alignment visualization. Availability:http://www.bioinformatics.org/strap/aa/ and http://strap.charite.de/aa/.Alignment-Annotator is a novel web service designed to generate interactive views of annotated nucleotide and amino acid sequence alignments (i)  Sequence alignment visualization is a widely used method to indicate differences and similarities in bio-polymers and to elucidate structurally and functionally important sequence positions. There already exist multiple web services that utilize or provide sequence alignments. Many of these applications offer their own implementation for server side alignment rendering, e.g. UniProt , Pfam 22, SCOPe Alignment-Annotator has been implemented in PHP, which uses the Java program Alignment-To-HTML to generate the HTML code . The cliIdentification of database IDs and homologous structures by the fast similarity/identity search program Blat (see below for more details).Caching of downloaded data and computational results.Holding of copies of the UniProtKB, the UniProt annotations, the PDB and the Catalytic Site Atlas on the server.Load balancing and process prioritization. This allows for termination of time consuming computations in favor of recently submitted jobs and the option to resume terminated jobs at a later time.Parallelization of data retrieval and computations.Collective item requests from remote services.A major challenge has been the computational time requirement for the generation of the alignment on the server side. Depending on the selected options, significant computations and external data retrieval may be involved. Therefore, we have rigorously optimized processing speed by the following strategies:The performance tuning was essential for usability and has significantly increased processing speed by 10-fold compared to the initial implementation.Submitted sequences undergo alignment computation unless the presence of gaps indicate that sequences are already aligned. By default, a rapid alignment method is used , whereasA total of eight different sequence alignment methods and five structure alignment programs can be specified in the first script, which is executed before alignment computation. Sequence based methods are usually sufficient if sequences are not too dissimilar. Mixed sequence/3D structure alignment can improve alignment quality of remote homologs and is provided as an option. However, 3D structure alignment requires more time, depending on the number and size of the 3D structures.The UniProt IDs of all input sequences are determined by perfect sequence match unless the sequence ID is explicitly specified in the input files or by script commands.We utilize Blat, which is about 50 times faster than Blast. It is optimized for databases that are small enough to fit into the main memory . It is uThe database identifiers are used to interrogate annotation servers. BioDAS is regarded as a standard for the exchange of residue specific information using XML ,15. The A simple and widely used method to display sequence annotations is to change the residue's foreground or background color. However, this does not allow discrimination of overlapping residue selections. This presents a real limitation as more and more sequence features are discovered and need annotation. Alignment-Annotator emphasizes residues by underlining. Text attributes and hyperlinks are exhibited in the balloon messages. This approach is more versatile and allows for multiple overlapping annotations. The layout of the residue selections is optimized for compactness. In order to avoid duplication of annotations, the server maps all possible annotation names to a controlled dictionary. Thereby equivalent annotations using different vocabulary e.g. Phosphoserine and Phosphorylated_serine are recognized and are merged. If an annotation constitutes a generalization of another one, the less precise is discarded in favor of the more precise annotation. For example, for the three annotations at the same sequence position, Phosphotyrosine, Modified_Residue and Post_translational_modification, only Phosphotyrosine is retained because it is most precise. In addition, the program performs a consistency check on all modified amino acids. For example, Phosphotyrosine has to be the letter Y.Alignment-Annotator is the first browser-based interactive alignment viewer, which runs on all platforms. Other programs can use Alignment-Annotator for visualization of dynamically generated alignments. Input data or a reference to the input data is submittable within the URL. For example, the browser page in Figure Alignment-Annotator allows user-defined residue annotations. For amino acid sequences, it displays annotations obtained from services and can derive information from PDB structure files. Data sources include several BioDAS services, the Catalytic Site Atlas  and the . heme in hemoglobin, residues in spatial neighborhood can be highlighted. Each sequence with a mapped 3D structure has a button to display that structure through the OpenAstex 3D-viewer. The default 3D-rendering style can be customized with the Strap scripting language in an implementation independent fashion. This retains the option to implement further 3D software in the future. The 3D style of selected amino acids can be changed by attaching 3D commands to residue annotations. Furthermore, several structures can be displayed together in one view and superimposed to compare their structures. Homologous PDB entries enable other knowledge data bases: The Catalytic Site Atlas is a manually curated collection of active site triads.When this option is activated, the most similar PDB entry of each sequence is used to obtain further residue specific information recorded in the PDB file. This is the secondary structure type such as α-helix, β-sheet, the residue number and chain letter and post-translational modification such as S-acetylation of cysteine. The secondary structure of one sequence in the alignment is displayed as a cartoon at the top. This is usually the one where the secondary structure spans the longest range. For 3D structures containing ligands, e.gAlignment-Annotator derives information from similar PDB structures even if the amino acid sequence is not identical. The information recorded in the PDB file or the Catalytic Site Atlas may not be correct for the sequence under investigation. This is because functional features of proteins like catalytic activity or ligand binding change during evolution. A well known text book examples are proteins of the eye lens, the crystallins, which are derived from enzymes but have lost catalytic activity. To emphasize potentially incorrect information, these annotations are labeled ‘By similarity’ as opposed to ‘By identity’ or the default evidence level. Annotations with adjacent sequence mismatches between the sequence and the 3D-structure are filtered out.Versatile commercial or free software for alignments exist (CLC-Workbench, Strap  and Jalvde novo.Alignment-Annotator is an alignment visualization platform for web services and for creating alignment views


Motivation: Antibody amino-acid sequences can be numbered to identify equivalent positions. Such annotations are valuable for antibody sequence comparison, protein structure modelling and engineering. Multiple different numbering schemes exist, they vary in the nomenclature they use to annotate residue positions, their definitions of position equivalence and their popularity within different scientific disciplines. However, currently no publicly available software exists that can apply all the most widely used schemes or for which an executable can be obtained under an open license.Results: ANARCI is a tool to classify and number antibody and T-cell receptor amino-acid variable domain sequences. It can annotate sequences with the five most popular numbering schemes: Kabat, Chothia, Enhanced Chothia, IMGT and AHo.Availability and implementation: ANARCI is available for download under GPLv3 license at opig.stats.ox.ac.uk/webapps/anarci. A web-interface to the program is available at the same address.Contact:deane@stats.ox.ac.uk The variable domains of antibodies and T-cell receptors (TCR) contain these proteins’ major binding regions. Alignment of these variable sequences to a numbering scheme allows equivalent residue positions to be annotated and for different molecules to be compared. Performing numbering is fundamental for immunoinformatics analysis and rational engineering of therapeutic molecules .Several numbering schemes have been proposed, each is favoured by scientists in different immunological disciplines. The Kabat scheme  is the sIn contrast to these Kabat-like schemes, IMGT  and AHo Separate online interfaces exist that can apply each numbering scheme: Kabat, Chothia and Enhanced Chothia through Abnum ; IMGT thWe have developed ANARCI, a program that can annotate sequences with all five of the numbering schemes described above. We provide both a web-interface and the software under open license so that these fundamental annotations can be easily available for further immunoinformatics analyses.ANARCI takes single or multiple amino-acid protein sequences as input. The program aligns each sequence to a set of Hidden Markov Models (HMMs) using HMMER3 . Each HMThe pre-aligned (gapped) germ-line sequences for the v-gene segment of each available species and domain type were downloaded from the IMGT/Gene Database . The seqAll possible pairwise combinations of the relevant v and j gene segments were taken to form a set of putative germ-line domain sequences. For the VH domain, the d gene segment was not included. Each position in the alignment represents one of the 128 positions in the IMGT numbering scheme.From the alignment an HMM is built using the hmmbuild tool. Here, the ‘—hand’ option is specified to preserve the structure of the alignment.The HMM for each domain type from each species was built in the following way:In total, 24 HMMs were built describing variable domain types from six different species. These HMMs were combined into a single HMM database using hmmpress.An input sequence is aligned to each HMM using hmmscan. If an alignment has a bit-score of less than 100 it is not considered further. This threshold proves effective at preventing the false recognition of other IG-like proteins. Otherwise, the most significant alignment classifies its domain type and the alignment is translated into a chosen numbering scheme.ANARCI can apply the Kabat, Chothia, Extended Chothia, IMGT or AHo schemes to VH, Vλ and Vκ domain sequences. The IMGT and AHo schemes can also be applied to Vα and Vβ domain sequences. Where possible, a position in the HMM alignment is annotated with the equivalent position in the numbering scheme. In regions where there is no direct equivalence between the alignment and the numbering scheme the sequence is numbered according to the specification described in the corresponding publication. For example, HMM alignment position 40 for a VH sequence is equivalent to Kabat position 31-35X depending on the length of CDRH1.For each numbered domain a header is written that describes the most significant alignment including the species, chain type and alignment range. The numbering follows in a column delimited format. Alternatively, users may import ANARCI as a Python module and use the API within their own scripts.With the rise of next generation sequencing, the ability to annotate large numbers of antibody sequences is becoming a common task. We used ANARCI to number a set of 1 936 119 VH sequences taken from a vaccination response study performed at Oxford University. The algorithm took three hours wall clock time using 32-cores with AMD Opteron 6272 Processors. All but 9560 sequences were successfully numbered. Where numbering failed the sequences had very unusual insertions or deletions that may be a result of sequencing errors.In addition to the command line tool we provide a webserver interface to ANARCI . Users mWe have developed ANARCI, a program for annotating antigen receptor variable domain amino-acid sequences with five commonly used numbering schemes. The program can be run as command-line tool or imported as a Python module for incorporation in custom scripts. We also provide a public web-browser interface that can annotate small numbers of sequences. ANARCI is freely distributed under the GPLv3 license and available at opig.stats.ox.ac.uk/webapps/anarci.


Computing alignments between two or more sequences are common operations frequently performed in computational molecular biology. The continuing growth of biological sequence databases establishes the need for their efficient parallel implementation on modern accelerators.This paper presents new approaches to high performance biological sequence database scanning with the Smith-Waterman algorithm and the first stage of progressive multiple sequence alignment based on the ClustalW heuristic on a Xeon Phi-based compute cluster. Our approach uses a three-level parallelization scheme to take full advantage of the compute power available on this type of architecture; i.e. cluster-level data parallelism, thread-level coarse-grained parallelism, and vector-level fine-grained parallelism. Furthermore, we re-organize the sequence datasets and use Xeon Phi shuffle operations to improve I/O efficiency.https://github.com/turbo0628/LSDBS-mpi.Evaluations show that our method achieves a peak overall performance up to 220 GCUPS for scanning real protein sequence databanks on a single node consisting of two Intel E5-2620 CPUs and two Intel Xeon Phi 7110P cards. It also exhibits good scalability in terms of sequence length and size, and number of compute nodes for both database scanning and multiple sequence alignment. Furthermore, the achieved performance is highly competitive in comparison to optimized Xeon Phi and GPU implementations. Our implementation is available at  Both tasks include iterative calculations of pairwise local alignments as a basic building block. This can lead to high runtimes for large-scale input data sets. Since biological sequence databases are continuously growing, finding fast solutions is of high importance. An approach to reduce associated runtimes is the implementation of basic alignment algorithms on parallel computer architectures –3. More Three levels of parallelization are required in order to exploit the compute power available in a cluster of Xeon Phis. Parallelization within a Xeon Phi is usually based on the “scale-and-vectorize” approach: scaling across the up to 61 cores requires the usage of several hundred threads while exploiting the 512-bit wide vector units requires SIMD vectorization within each core. Recent examples of efficient parallelization on Xeon Phis include scientific computing , and datThe rest of this paper is organized as follows. The “Xeon Phi is a coprocessor connected via the PCI express (PCIe) bus to a host CPU. From a hardware perspective, it contains up to 61 86 compatible cores. Each core features a 512-bit vector processing unit (VPU) based on a new instruction set. The cache hierarchy contains a L1 data cache of size 32 KB and a 512 KB per core L2 cache. The cores are connected via a bidirectional ring bus which enables L2 cache coherence based on a directory based protocol. Each core can execute up to four threads at the same time.Assuming a Xeon Phi with 61 usable cores running at 1.238 GHz, we can determine the peak performance for 32-bit integer  operations as follows: 16 (#SIMD lanes) × 1 integer operation × 1.238 GHz × 61 (#cores) = 1.208 Tera integer operations per second.From a software perspective, three programming models can be used in order to harness the compute power of the Xeon Phi: (i) native model, (ii) offload model, and (iii) symmetric model. In this paper, we have chosen the offload model. In this model, code sections and data can be offloaded from the host CPU to the Xeon Phi. Using OpenMP pragmas, offload regions can be specified. When encountering such a region during program execution, the necessary data transfers between host and Xeon Phi are performed and the code inside the  region is executed on the Xeon Phi.The database search application considered in this paper scans a protein sequence database using a single protein sequence as a query (similar to BLASTP). Different to the BLASTP heuristic, we calculate the score of an optimal local alignment between the query and each subject sequence using the Smith-Waterman algorithm with affine gap penalties (instead of a seed-and-extend approach). The subject sequences are ranked in terms of this score. Actual alignments are only computed for the top-ranked database sequences which only takes a negligible amount of time in comparison to the score-only search procedure. Note that the score-only Smith-Waterman computation can be performed in linear space and quadratic time with respect to the length of the alignment targets.Q and S and length q and s, respectively. We want to compute the score of an optimal local alignment of Q and S with respect to a given scoring scheme consisting of a gap opening penalty α, a gap extension penalty β and an amino acid substitution matrix sbt. The well-known Smith-Waterman algorithm solves this problem by computing a dynamic programming matrix iteratively based on the following recurrence relations: Consider two protein sequences HA=HA=E=F=0 for all 0<=i<=q, 0<=j<=s.The iterative computation of theses matrices is started with the initial values: The time complexity of computing an optimal multiple alignment of more than two sequences grows exponentially in terms of the number of input sequences. Thus, heuristic approaches with polynomial complexities must be used in practice for large inputs to approximate the  optimal multiple alignment.Distance matrix: For each input sequence pair, a distance values is computed based on the Smith-Waterman algorithmGuide tree: Using the distance matrix computed in the previous step is taken as an input to compute an evolutionary tree using the neighbor-joining method [g method .Progressive alignment: Following the branching order of the tree a multiple sequence alignment is build progressively.The multiple (protein) sequence alignment application considered in this paper is the first stage of the popular ClustalW heuristic . ClustalWe briefly review some previous work on accelerating pairwise alignment (based on Smith Waterman) and progressive multiple sequence alignment  on a number of parallel computer architectures. A number of SIMD implementations have been designed in order to harness the vector units of common multi-core CPUs e.g. –21) or t or t21])The solution in this paper is based on a cluster of Xeon Phis. Compared to common CPUs, a Xeon Phi contains significantly more cores and often a wider vector unit. Different from CUDA-enabled GPUs, a Xeon Phi provides x86 compatibility, which often simplifies the implementation process. Nevertheless, achieving near-optimal performance is still a challenge which needs to be addressed by parallel algorithm design and efficient implementation. In this paper we demonstrate how this can be done for protein sequence database search and distance matrix computation for multiple sequence alignment.We have designed new algorithms which can handle searching tasks for large-scale protein databases on Xeon Phi clusters.We have designed new algorithms for calculating large-scale multiple sequence alignments on Xeon Phi clusters.We have implemented our multiple sequence alignment algorithm using the offload model to make full use of the compute power of both the multi-core CPUs and the many-core Xeon Phi hardware.Compared to our previously presented LSBDS , we intrWe have observed two facts: (1) protein sequence database search has inherent data parallelism; (2) each VPU on Xeon Phi can execute multiple integer operations in an SIMD parallel way efficiently. Based on these two facts, we have partitioned the database search process on a single node into two data parallel parts: device level and thread level. The device level data parallel part is encoded on the host CPU. It splits the subject database into multiple batches that can be distributed to CPU and Xeon Phi devices. The thread level data parallel part is used to process data batches locally. In order to support search tasks for large-scale databases, we have designed a dynamic data distribution framework to distribute these batches to both the host CPU device and the Xeon Phi devices. In order to solve the performance loss problem for searching long query sequences, we have also proposed a multi-pass algorithm where long query sequences are partitioned into multiple short subsequences for consecutive searching passes. We have presented more implementation details of our algorithm in .The distance matrix computation stage of ClustalW is typically a major runtime bottleneck. Thus, in our work we have only concentrated on designing a parallel algorithm for this stage. ClustalW bases the distance computation between two protein sequences on the following concept :Si,Sj∈S={S1,…Sn}. The following equation defines their distance d: Consider two sequences nid is defined as the number of exact matches in an optimal local alignment between Si and Sj. li (lj) is the length of Si (Sj).whereby nid can be calculated in the Smith-Waterman traceback procedure by counting the number of exact character matches. Figure nid-value computation on the Xeon Phi architecture. That is we have used the following definition and theorem to calculate the nid-value without doing the actual traceback.The value S1 and S2, affine gap penalties α, β, and substitution matrix sbt. The matrix NA  is defined in terms of the following recurrence relations: Consider two protein sequences where imax,jmax) denote the coordinates of the maximum value in the corresponding pairwise local alignment DP matrix HA.It can be shown that Si to S={Si+1) making the subject sequence set for distance matrix computation comparatively small. In order to design an efficient parallel distance matrix computation algorithm on Xeon Phi, we have used the task partitioning method shown in Fig. sprofile and the mprofile in Fig. m (see Definition 2) values in parallel. Figure sprofile and the mprofile. VPUs will make use of these two score vectors to load substitution values and m values quickly. The shuffling procedure in Fig. Before performing the alignment process, two temporary score vectors : Partitions subject database or MSA tasks into a number of chunks in a preprocessing steps and sends them to compute nodes.Algorithms on a Single Node (Worker): Receives sequence chunks from master and performs the corresponding DP calculations.Result Collector (Master): Performs additional operations required to further process the returned results.Our approach is based on the fact that both subject database batches (for database searching) and MSA tasks can be scanned in parallel. Thus we have implemented the cluster level data parallel algorithm for these two alignment applications. The cluster level data parallel algorithm is encoded on the master node. The master-node partitions the subject database or the MSA tasks into a number of chunks that will be sent to different compute nodes. Our approach is implemented using the following modules: Pi for node i. With the performance factor Pi, we can then calculate the appropriate size of the database subset allocated to node i.In our work, we have implemented a static dispatcher for our cluster level parallel database searching algorithm. Figure We have designed and implemented a cluster level dynamic dispatcher to distribute tasks to compute nodes. Figure -Intel Xeon Phi 7110P: 61 hardware cores, 1.1 GHz processor clock speed, 8 GB GDDR5 device memory.-Intel Xeon Phi 31S1P: 57 hardware cores, 1.1 GHz processor clock speed, 8 GB GDDR5 device memory.We have implemented the proposed methods using C++ and evaluated them on compute nodes with the following Xeon Phi cards (with ECC enabled) installed: Tests have been conducted on a Xeon Phi cluster with three compute nodes that are connected by an Ethernet switch. There are two Xeon E5 CPUs and 16GB RAM on each compute node. The cluster runs Centos 6.5 with the Linux kernel 2.6.32-431.17.1.el6.x86_64. The CPU configuration on each node varies, as is listed in Table cell updates per second (CUPS). A CUPS represents the time for a complete computation of one entry of the DP matrix, including all comparisons, additions and maxima operations.A performance measure commonly used in computational biology to evaluate Smith-Waterman implementations is We have scanned three protein sequence databases: (i) the 7.5 GB UniProtKB/Reviewed and Annotated , (ii) the 18 GB UniProtKB/TrEMBL , and (iii) the 37 GB merged Non-Redundant plus UniProtKB/TrEMBL  for query sequences with varying lengths. Query sequences used in our tests have the accession numbers P01008, P42357, P56418, P07756, P19096, P0C6B8, P08519, and Q9UKN1.We have firstly compared the single-node performance of our methods to SWAPHI  and CUDAN2 node (see Table α=10, and β=2 have been used in our experiments. The substitution matrix used is BLOSUM62.For single-node tests, we have used the computing CUPS values in our experiments. Figure 1010We have measured the time to compute the similarity matrices to calculate the SWAPHI and CUDASW++ 3.1 cannot support search tasks for the 18 GB and 37 GB databases. Thus, we only use our methods to search them. Figure 10Figure 10S1 to S6 are used to compare the performance of our method and MSA-CUDA, where the sequence numbers are small since MSA-CUDA can not handle datasets with large sequence number. Datasets L1 to L6 are used to evaluate the performance of our method for handling large-scale datasets. These datasets consist at least 10,000 sequences.A set of performance tests have been conducted using different protein sequence datasets to evaluate the processing time for the distance matrix computation step of our implementation in comparison to MSA-CUDA . The datLi denotes the length of the ith sequence in the dataset. Thus, the workload W is actually the total number of matrix cells to be calculated. As our method utilizes the constant 25 instructions for calculating each cell (as is listed in Fig. W. Table The workload for computing a distant matrix grows quadratically with respect to the number of input sequences. The average sequence length of the dataset also has a great impact on the computing workload. We have used the following equation to measure the workload needed to process a dataset. entclass1pt{minimaS1 to S6, MSA-CUDA is benchmarked on a Tesla K40 GPU with default options and all available compiler optimizations enabled. Our implementation runs on an Intel Xeon Phi 7110P with 240 threads. Figure For the medium-scale datasets L1 to L6, MSA-CUDA cannot work normally. We have run our methods on a single Intel Xeon Phi 7110P, the N2 node and the cluster respectively. The performance results are shown in Fig. For the large-scale datasets We have presented two parallel algorithms for protein sequence alignment based on the dynamic programming concept which can be efficiently mapped onto Xeon Phi clusters. Our methods exhibit good performance on a single compute node as well as good scalability in terms of sequence length and size, and number of compute nodes for both protein sequence database search and distance matrix computation employed in multiple sequence alignment. Furthermore, the achieved performance is highly competitive in comparison to other optimized Xeon Phi and GPU implementations. Biological sequence databases are continuously growing establishing the need for even faster parallel solutions in the future. Hence, our results are especially encouraging since performance of many-core architectures grows much faster than Moore’s law as it applies to CPUs. For instance, the performance improvement with at least a factor of 3 can be expected on the already announced next-generation Xeon Phi product.