Driver’s gaze information can be crucial in driving research because of its relation to driver attention. Particularly, the inclusion of gaze data in driving simulators broadens the scope of research studies as they can relate drivers’ gaze patterns to their features and performance. In this paper, we present two gaze region estimation modules integrated in a driving simulator. One uses the 3D Kinect device and another uses the virtual reality Oculus Rift device. The modules are able to detect the region, out of seven in which the driving scene was divided, where a driver is gazing at in every route processed frame. Four methods were implemented and compared for gaze estimation, which learn the relation between gaze displacement and head movement. Two are simpler and based on points that try to capture this relation and two are based on classifiers such as MLP and SVM. Experiments were carried out with 12 users that drove on the same scenario twice, each one with a different visualization display, first with a big screen and later with Oculus Rift. On the whole, Oculus Rift outperformed Kinect as the best hardware for gaze estimation. The Oculus-based gaze region estimation method with the highest performance achieved an accuracy of 97.94%. The information provided by the Oculus Rift module enriches the driving simulator data and makes it possible a multimodal driving performance analysis apart from the immersion and realism obtained with the virtual reality experience provided by Oculus. Event simulation is increasingly being used in many different areas. Simulators can speed up the process of acquisition of basic abilities and are configured as tools of great learning and research capacity in a wide range of areas such as surgery  or InterWith that aim, we developed a driving simulator with varied routes (urban and interurban) and traffic events to analyze the driving safety level. The simulator can show in real time and store in files for further processing, not only information about the vehicle , gear, and fuel consumption) but also any committed traffic violations. We also developed an Android application that controls the Shimmer physiological sensors (electrocardiogram (ECG), electromyogram (EMG), and galvanic skin response (GSR)) through a Bluetooth connection . The AndAn important aspect to measure driver allocation of attention is the gaze location and patterns ,6. Gaze The Kinect sensor add-on for the Xbox 360 video game platform, which emerged at the end of 2010, interprets the 3D information of the scene obtained through infrared structured light that is read by a standard CMOS sensor. Its sampling frequency rate is 30 fps (frames per second). It is a low-cost alternative to 2D cameras that present problems difficult to address, such as occlusions between human body parts, range of movements, and drastic illumination and environmental changes. Since its commercial launch, many developers and researchers have used the Kinect device in their work, in different areas, e.g., head-pose and facial expression tracking, hand gesture recognition, human activity recognition, and healthcare applications ,26. JafaVirtual reality (VR) is common in video games and is increasingly present in other areas such as education and medicine . VR can An outstanding virtual reality device is the Oculus Rift . Oculus Rift is an inexpensive 3D VR HMD released in 2012. It has 110° FOV in diagonal, a combined resolution of 1280 × 800, and an update frequency of 60 Hz. Oculus embeds micro-electrical-mechanical (MEMS) sensors and can track and monitor head movements so that the content display can be compensated in an immersive VR environment . AlthougIn the previously cited works using the Kinect ,28,29, tThe rest of the paper is organized as follows: We aimed to integrate a gaze region estimation module in a driving simulator that we had already developed. The driving simulator includes different scenarios and traffic situations interesting to analyze the safety level of the drivers in the experiments. The driver has to react to all the situations adequately to ensure safety. The simulator stores a lot of data from the driver vehicle on each route such as position, speed, rpm, gear, and fuel consumption. The simulator was developed with the Unity game engine  and the 3D modeling programs Blender  and 3ds Max . It can be executed in an off-the-shelf computer and a low-cost input device such as the G27 . The simulator allows to analyze the driving style of different people taking into account their reactions to the traffic situations they have to face and their outcomes. It also allows the awareness-raising and the learning of safe driving techniques.The joint analysis of the gaze patterns, driving styles, and safety level of drivers can identify the visual search strategies that lead to the safest driving. Moreover, the training on these visual search strategies included in the simulator can improve visual search skills in drivers of varied age and experience as stated by .Driver’s gaze can be characterized through different representations such as directional vector , 3D poinWe have adopted two different approaches to estimate the gaze region in the simulator, each one with a different visualization device. First, we use a big screen of dimensions 260 × 195 cm as the visualization device and the 3D Kinect device to estimate the gaze region. Then, we use the Oculus Rift device as both the visualization and the gaze estimation device.For the Kinect, we have used the Kinect for Windows SDK and the Face Tracking SDK for Kinect for Windows in the driving simulator Unity project to integrate a module in our simulator to achieve Kinect-based head tracking and gaze region estimation from it.When a user is in front of the Kinect device, the gaze estimation module achieves head tracking through the extraction of a large number of facial points. If these points are joined forming triangles, the 3D facial mesh is built. The module tracks robustly the head in frontal and non-frontal positions. From those facial points, three rotation angles are obtained: yaw, pitch, and roll. Yaw is the head rotation around the vertical axis, pitch is the head rotation around the side-to-side axis, and roll is the head rotation around the front-to-back axis.We estimate the gaze region through the head position. From experimental results, we discarded the roll angle because it does not provide relevant information regarding the driver’s head position when he or she is gazing at the different regions in which we have divided the simulator scene. The yaw and pitch angles are the basic values used for gaze estimation in the different adopted methods, which are explained in the next section.With Kinect, the simulator scene is projected onto a big screen of dimensions 260 × 195 cm. The distance between the screen and the driver is 250 cm. The increase of the yaw angle to move the head from the place when the user is gazing at the central position of the screen, in order to gaze at the right border of the screen, is 27.47 deg supposing that the eyes remain in the same position that they have while gazing at the central position of the screen. That value can be obtained taking into account that the tangent of this angle equals the division between half the screen width and the distance between the screen and the user see . ConsequWe will use numbers 1 to 7 to refer to these regions throughout the paper. It was very important to take a good decision about the number of scene regions as it is necessary to distinguish among all the regions that the driver is going to gaze for a different purpose but taking into account that the larger the number of regions is, the worse the gaze region estimation performance is. Many research studies used similar gaze regions both in simulation and on-road studies with small differences to adapt to the particular environment and FOV ,44,45,46With Oculus as a visualization device, the scene is rendered twice, one with the left camera and another with the right camera, in such a way that each eye watches the corresponding scene, producing the three-dimensional scene with the fusion of both scenes inherent in human stereoscopic vision. We have integrated an Oculus-based gaze region estimation module in our driving simulator that uses the information of the yaw and pitch head rotation angles. The scene is divided in the same seven regions previously mentioned when the visualization device is the big screen. We have implemented four different methods for gaze estimation and have applied them using the Kinect device and the Oculus Rift.This method uses the yaw and pitch angles. Roll was discarded as it is a movement not naturally made in a driving scenario as mentioned in . On the Both yaw and pitch initially range from 0 to 360 deg increasing rightwards and upwards. We have subtracted 360 deg to the value of yaw and pitch if they range from 180 to 360 deg. This way, they range from −180 to 180 deg. The pitch angle has a positive value if the head is looking up the ceiling and has a negative value if the head is looking down the floor. The yaw angle has a positive value if the head is turned towards the right shoulder and it has a negative value if the head is turned towards the left shoulder. Although absolute values near 90 deg are difficult to track, the range of values for a person driving in the simulator are much smaller and then precisely obtained by the Kinect.yaw (x axis) and pitch (y axis) are transformed, the displacement in the x axis  in the screen as shown in The values of The values of h is the height of the screen.Similarly, the values of After the calibration, the value of Depending on This method is a modification of the first method, in which not only the relation between the head movement and the gaze displacement is considered, but also the different way the users are gazing at the different regions. This problem was considered following a statement present in , which iIn this method, the calibration stage to configure the decision maker consists in gazing at a series of points distributed along the border of the different regions in which we have divided the scene, as shown in Once calibration with all the points is fulfilled, the decision maker is built comparing In this method, we adopted a multi-layer perceptron (MLP), which has been successfully applied to a great variety of problems ranging from time series forecasting to image classification. MLP is a feedforward artificial neural network (ANN) whose units or neurons are organized in layers: one input layer, one or more hidden layers, and one output layer. The values of the features of an input pattern are transferred from the input layer to the hidden layers and then to the output layer in a feedforward fashion. The layers are fully connected and the value of each neuron depends on the value of the neurons of the previous layer, the weights of the connections with that layer, and an activation function . The mosWe adopted the C++ implementation of MLP provided by the Machine Learning Library (MLL), which is included in Open Source Computer Vision Library (OpenCV). We have used an architecture comprising of seven binary MLPs in the method. All the MLPs have a hidden layer comprised of 14 neurons. Each MLP is in charge of detecting that the user is gazing at one, out of the seven regions of the scene. Once a pattern is processed for the seven MLPs, the detected gaze region will be given by the MLP with the largest output value.(1)Yaw.(2)Pitch.(3)Displacement in the x axis of the central point of the rectangle given by the face detection in each frame. This displacement is relative to the value in the x axis of the central point of the rectangle given by the face detection when the user is asked to gaze at the central point of the scene.(4)Displacement in the y axis of the central point of the rectangle given by the face detection in each frame. This displacement is relative to the value in the y axis of the central point of the rectangle given by the face detection when the user is asked to gaze at the central point of the scene.(5)Area of the rectangle given by the face detection in each frame.As input parameters to each MLP, we selected the parameters used in , namely:J5(ξ) criterion [J5 (ξ) is the trace (tr) of the matrix result of the product of the inverse of wS, which is the average of the covariance matrices inside each class, and the bS, which is the covariance matrix for the averages of each class:With these five features, we applied a feature selection algorithm to analyze the discriminative power of them. We adopted the riterion . FollowiThe classification of the features using this criterion from the most to the least discriminative is: 1, 2, 3, 4, and 5 (preserving the number in which they were previously mentioned) with weights 36.44, 68.35, 71.36, 72.85, and 74.23, respectively. The first weight represents the discriminative power of the first feature, the second weight represents the discriminative power of the first and second features, the third weight represents the discriminative power of the first, second, and third features, and so on.To generate the patterns for the MLP training in the calibration stage, each user has to gaze at each region 10 s moving their eyes over the region to generate 150 patterns. Each pattern has the previously mentioned five features: yaw, pitch, displacement in the x axis of the central point of the rectangle given by the face detection in each frame, displacement in the y axis of the central point of the rectangle given by the face detection in each frame, and area of the rectangle given by the face detection in each frame. After gazing at each region, the user has to gaze at the central point of the scene for two reasons: firstly, to consider the influence of the initial position of the head on its movement to gaze at another region, and secondly, to calculate the mean value of the Maximum value of iterations: 1000.Convergence parameter epsilon: 0.001.Training method: Backpropagation.Strength of each gradient in the weight computation: 0.1.Difference between weights of two consecutive iterations: 0.1.The configuration of the seven MLPs of the adopted architecture are similar. The value of their training parameters is:The selected activation function is the symmetric sigmoid.The classifier used in this method is the support vector machine (SVM). It is a pattern classification technique applied in many different machine learning areas, such as generalized predictive control and image classification . Due to Maximum value of iterations: 100,000.Convergence parameter epsilon: 0.001.Similarly to the implementation of MLP, we adopted the C++ implementation of SVM provided by the MLL, which is included in OpenCV. The steps for the creation and configuration of the classifier are similar to the steps in method 3, although, instead of using seven classifiers in the system, it is comprised of just one SVM classifier, which has as output the region, out of the seven, where the user is gazing at in each frame. Thus, to generate the patterns for the SVM training in the calibration stage, each user has to gaze at each region 10 s moving their eyes over the region to generate 150 patterns. Each pattern has the same five features used for method 3: yaw, pitch, displacement in the x axis of the central point of the rectangle given by the face detection in each frame, displacement in the y axis of the central point of the rectangle given by the face detection in each frame, and area of the rectangle given by the face detection in each frame. These 150 patterns per region and per user are used to train the SVM. The selected training parameters are:Parameter controlling the outlier separation: 1.Kernel type: Linear.The selected classifier parameters are:We evaluated the Kinect-based and the Oculus-based gaze region estimation using the four methods explained in the previous section. Twelve subjects took parts in the experiments. The mean age of the subjects ranged from 20 to 55, with a mean of 27.83. The mean years of driving experience was 8.75 years, ranging from less than a year to 38 years. Seven of the subjects had experience as video game players while the remaining five subjects did not have such experience.t-tests [Many significant research studies about driver gaze region estimation carried out their experiments with 12 or fewer subjects, such as ,44,45,53t-tests  using thEach driver made the same route in our driving simulator in the research laboratory twice: first using a screen as the visualization display and another time using Oculus Rift as the visualization display. We decided that users would use Oculus Rift the second time they drove on the route. With VR devices such as Oculus Rift, simulator sickness has been reported that could force users to withdraw from the experiments . ConsequApproximately 500 pairs of regions were collected for each participant and method in the Kinect-based gaze estimation and another 500 pairs in the Oculus-based gaze estimation. Each pair is composed by the actual region the participant was gazing at and the region predicted by the method for the same image. The testing set is composed by the patterns formed from the processing of the images corresponding to the moments when each participant had to say out loud the number of the region that was gazing at every 4 s. These patterns have 2 features  and thus the actual regions the drivers were gazing at are not equally distributed. The region 1 (straight ahead) was by far the most gazed at region and regions 3 (rear-view), 4 (left mirror and left door glass), and 5 (steering wheel with another speedometer) were more gazed than regions 2 (speedometer), 6 (rpm meter and gear indicator), and 7 (right part of the dashboard). The regions more present in the testing set were the regions more important to drive properly on the simulator route. To obtain the overall accuracy of each method, we considered all the images equally in the testing set, instead of averaging the accuracy for all the regions. Therefore, the gazed regions more present in the images, as they are more important for the driving, had more influence on the overall accuracy. We aimed to compute the performance of the methods this way as the usefulness of the gaze region estimation module in the driving simulator would then be higher in further experiments to analyze the gaze patterns of drivers as a function of their features and performance. For this reason, we discarded to use in the testing set images where drivers were gazing at all the regions with the same frequency as this approach would have not been representative of real and natural driving.Before analyzing the Kinect-based and Oculus-based performance of the different methods, we provide some statistics about yaw and pitch of the users while driving in the experiments. In the next subsections, first, the results for the four methods using the Kinect are presented. Then, the same methods using the Oculus Rift are presented.The confusion matrices for the 7-region gaze estimation problem using each of the four developed methods and for all the users are presented in Using the bidimensional method with static decision-maker (method 1), the global accuracy including all the regions is 88.79%. All the regions except region 3 have an accuracy higher than 84%. The regions with higher accuracy are region 1, which has the largest area and is on the center of the image, and regions 2, 6, and 7, which are relatively large and placed on the corners of the scene. From these four regions, region 7 has the highest accuracy. The region with the lowest accuracy is region 3, which is the smallest region and farthest from the center of the scene. The individual results of three users yield interesting results: the user that mostly accompanied the gaze displacement with the head movement, the user with average head movement, and the user that accompanied the gaze displacement with the smallest head movement. The results of the user with the biggest head movement were coherent with the overall results. Although his/her overall accuracy is considerably larger (94.92%), the smallest accuracy percentage for region 3 (74%) is quite large, being the smallest and most cornered region. Accuracy in region 4 is considerably large (97%) compared to the accuracy in region 4 for all the users (84%), due to the pronounced head movement when he/she was looking gazing at this region. The results of the user with the average generic head movement are similar to the results considering all the users, although the regions 2 and 5 have particularly good results (95%). The user with the smallest head movement has an overall accuracy of 84.84% with particular bad results in region 3 (44%). It can be observed that users with small head movements have large error rates in region 3. The second region with the largest errors is region 4 as it is quite small and has several surrounding areas.Using the bidimensional estimation with adapted decision maker (method 2), the global accuracy is 92.85%, 4.06% larger than with method 1. Moreover, the errors are distributed more regularly among all the regions. The regions with the smallest error are regions 1 and 5, which are on the center of the scene. The error rate of the user with the largest head movement is below 4%. The only region with accuracy smaller than 96% for this user is region 3, with 84%. The global accuracy of the user with average head movement is even better (96.61%) than the accuracy of the user with the biggest head movement. Method 2 approximates the results of all the users achieving a large overall accuracy. The calibration stage in this method is better adapted to each user as the different way a user moves his/her head to gaze at each region is considered. Region 3 is not the one with the smallest accuracy as regions 4 and 6 have smaller accuracy. The accuracy of the user with the smallest head movement has increased 4.45% from method 1. Regions 3 and 4 are by far the ones with the smallest accuracy (59% and 69%) as they are small and the user made a small head movement to gaze at them.With the MLP classifier (method 3), the global accuracy is 64.79%, much worse than the accuracy obtained in the first two methods. In general, the detection error in all the regions is quite similar, so the classifier is balanced. Differently from the results in methods 1 and 2, there are errors between not adjacent regions. These errors should be caused by outliers in the training stage. Regarding the results from the user with the largest head movement, the user with average head movement, and the user with the smallest head movement, they follow the expected trend although there are not satisfactory results in any of them. We analyzed the performance of the MLP-based classifier using only the yaw and pitch features, similarly to methods 1 and 2. The results are shown in With the SVM classifier, the global accuracy is 64.73%, very similar to method 3. The regions with the largest detection error are 4 and 6, when these regions in method 3 are 1 and 4. The accuracy for the user with the largest head movement is 87.60%. The region with the largest detection error (27%) is region 2, which is a region that is not frequently gazed at. The accuracy for the user with average head movement is 53.01%, worse than the overall accuracy considering all the users. The user with the smallest head movement has a very large error rate (50.61%), even larger than the accuracy. We tested the SVM system using only the features yaw and pitch . The oveIn summary, method 2 is the most powerful method as it has an easy calibration and the best results. After that, method 1 is the second best. The third is the SVM system as it can work with relatively good results with only two features. The worst method is the MLP-based system although it has a considerable ability to adapt to any type of user.The confusion matrices for the 7-region gaze estimation problem using each of the four developed methods and for all the users are presented in Using the bidimensional estimation with adapted decision maker (method 2), the global accuracy is 97.94%, better than in method 1. Moreover, region 4 has the smallest accuracy (89%) but with an increase of 8% with respect to method 1. The accuracy is 5.09% larger than in method 2 using the Kinect.Using the MLP classifier (method 3), the global accuracy is 59.95%, much worse than in methods 1 and 3. Moreover, the accuracy is 4.84% worse than the result of the same method using Kinect unlike the comparison of methods 1 and 2 using the Kinect and Oculus. The largest region (1) has worse accuracy (55%) than smaller regions such as 3, 4, and 6.Using the SVM classifier (method 4), the global accuracy is 87.48%, much better than method 3 but worse than methods 1 and 2. Differently to what happened with method 3 comparing the use of the Kinect and Oculus, the overall accuracy of method 4 using the Oculus is 22.75% better than using the Kinect.The Kinect-based gaze region estimation module processes 30 fps, whereas the Oculus-based gaze region estimation module processes 60 fps. The dissimilar frame rates are given by the features of the Kinect and Oculus Rift devices. This difference did not affect the comparative analysis of both modules as the performance tests are based on the results given by the developed methods (the predicted region) and the ground-truth  every 4 s regardless of the frame rate of each device.FOV and mostly the distance between regions are larger with the Oculus than with the screen as visualization device. Consequently, head movements to gaze at the different regions are larger, which had some influence on the better performance of the methods with the Oculus than with the Kinect with all the users regardless the degree of head movement in each gaze displacement and the use of peripheral vision. The results are also caused by the precision of the head orientation given by the MEMS sensors embedded in the Oculus Rift and by the way the users move their eyes and head in a VR environment such as the Oculus Rift in both the calibration stage and on route, more similarly than using the screen as the visualization device.The system that uses seven MLPs is the only method that has worse performance with the Oculus. A change in the configuration of the MLP-based system may have changed the performance although we have maintained the same configuration used with the Kinect to make fair comparisons. Training patterns and training process are fundamental in the performance of the MLP and SVM methods. The performance of methods 1 and 2 are quite sound with a difference in accuracy for region 4 , sensitively better in method 2. Calibration takes more time and is more complex in methods 3 and 4 than in methods 1 and 2, which also have the best performance.To assess the Oculus-based driving simulation, the users were asked to rate two aspects: the level of immersion and realism and the level of smooth natural head tracking provided by the Oculus Rift. The users had to rank both aspects quantitatively ranging from 0 (the worst rating) to 10 (the best rating). The average score for the two aspects was very satisfactory: 8 for the level of immersion and realism and 9 for the level of smooth natural head tracking.p 0.05). Therefore, there is no difference between G1 and G2 in the variance of the subtraction between the Kinect-based gaze estimation and the Oculus-based gaze region estimation accuracies and the two groups are homogeneous with respect to the computed variable. In the ANOVA analysis, the significant level p-value was 0.744 (p 0.05) so there are no significant differences between the value of the subtraction between the Kinect-based gaze estimation and the Oculus-based gaze region estimation accuracies between the two groups. This way, it was proved that the order in the use of the gaze estimation device did not affect the obtained gaze region estimation accuracies.As it was already stated, the order of the devices used in the experiments was the same for all the subjects: first with the Kinect and using a screen as the visualization display and later using the Oculus Rift as the visualization display. Although we did it to avoid possible simulation sickness using Oculus, the order of the experiments may have affected the results. Using first the Oculus and then Kinect in the experiments for a subset of the subjects may have led to different gaze region estimation accuracies than using first the Kinect and then the Oculus. To test the possible influence of the order of the experiments, we have repeated the experiments with a subset of seven of the available subjects that took part in the already presented results. In the repeated experiments, the subjects drove in the same scenario twice, first using the Oculus Rift to estimate the gaze region and then using the Kinect. We have fulfilled a one-way analysis of variance (ANOVA) using the SPSS statistical package to test the influence of the order of the experiments in the subtraction between the Kinect-based and the Oculus-based gaze region estimation accuracies. Let G1 be the groups of experiments using first the Kinect and then the Oculus and let G2 be the groups of experiments using first the Kinect and then the Oculus. Before the ANOVA test, Levene’s test for equality of variances was performed. The obtained significance level p-value was 0.439 (We have developed Kinect-based and Oculus-based gaze region estimation modules integrated in a driving simulator so that gaze region can be stored in the simulator and used in the analysis together with other data from the vehicle and the driver. We carried out a novel and original comparative analysis between both modules. Four different methods were implemented and compared to estimate gaze region through head movement in both modules. The Oculus Rift achieved an overall better performance than the Kinect due to different reasons such as the larger distance between regions with the Oculus, the precision of the head orientation from MEMS sensors, and the different features of head movements the drivers made to gaze at the different scene regions. The Oculus-based gaze region estimation method with the highest performance uses a bidimensional estimation with adapted decision maker. This method achieved an accuracy of 97.94% and outperformed MLP and SVM. The Oculus Rift, which makes driving experience more immersive and realistic, can be used without any additional hardware to obtain the region the driver is gazing at every frame in future research studies to analyze the gaze pattern of drivers as a function of their features and correct them reeducating the drivers if these patterns are not suitable for safe and efficient driving. The Oculus-based gaze region estimation provided by the method with the highest performance can be applied to other areas where virtual reality is used such as education or clinical therapies. The extraction of the region the user is gazing at while using a virtual reality application enriches the computed information and enables further analysis relating his/her performance in a VR application and his/her visual attention to the different stimulus present in the VR scenario.Our approach does not obtain eye tracking to estimate gaze position but head tracking. We considered that eye tracking requires restrictions that cannot always be met in vehicular environments. Moreover, some virtual reality devices such as the Oculus Rift do not support eye tracking and the necessary introduction of other device providing eye tracking between the user and the Oculus Rift would greatly complicate its applicability and adoption. Therefore, the use of head movement to obtain gaze data is particularly interesting in driving and virtual reality environments. To achieve a precise gaze region estimation from head tracking, a calibration stage is included in our modules to adapt to the individual relation between head rotations and gaze displacements. This calibration process is fast and has to be carried out only once even though a driver uses the simulator repeatedly.After the validation of the Oculus-based gaze region estimation module, we will use it in our diving simulator for experiments with many different scenarios and drivers to analyze the relation between the profile of the drivers, their gaze patterns, reactions to significant traffic events, and level of safe and efficient driving. We plan to adapt the Oculus-based gaze region estimation approach to other VR applications we developed in the area of physical and cognitive rehabilitation to enrich the extracted information from the use of the applications with the patients’ gaze patterns.


The alignment of character sequences is important in bioinformatics. The quality of this procedure is determined by the substitution matrix and parameters of the insertion-deletion penalty function. These matrices are derived from sequence alignment and thus reflect the evolutionary process. Currently, in addition to evolutionary matrices, a large number of different background matrices have been obtained. To make an optimal choice of the substitution matrix and the penalty parameters, we conducted a numerical experiment using a representative sample of existing matrices of various types and origins.Accuracy, Confidence) on the evolutionary distance between sequences and the evolutionary distance to which the substitution matrices correspond. Optimization of a combination of matrices and the penalty parameters was carried out for local and global alignment on the values of penalty function parameters.We tested both the classical evolutionary matrix series ; structural alignment based matrices, contact energy matrix, and matrix based on the properties of the genetic code. This study presents results for two test set types: first, we simulated sequences that reflect the divergent evolution; second, we performed tests on Balibase sequences. In both cases, we obtained the dependences of the alignment quality (Consequently, we found that the best alignment quality is achieved with matrices corresponding to the largest evolutionary distance. These matrices prove to be universal, i.e. suitable for aligning sequences separated by both large and small evolutionary distances. We analysed the correspondence of the correlation coefficients of matrices to the alignment quality. It was found that matrices showing high quality alignment have an above average correlation value, but the converse is not true.This study showed that the best alignment quality is achieved with evolutionary matrices designed for long distances: Gonnet, VTML250, PAM250, MIQS, and Pfasum050. The same property is inherent in matrices not only of evolutionary origin, but also of another background corresponding to a large evolutionary distance. Therefore, matrices based on structural data show alignment quality close enough to its value for evolutionary matrices. This agrees with the idea that the spatial structure is more conservative than the protein sequence. Alignment is the most common bioinformatics procedure.A natural quality criterion for the alignment procedure is the reproducibility of true alignment, i.e., the restoration of true events at the level of substitutions and insertion-deletions of amino acids in a symbolic sequence. To be precise, as applied to the scheme of divergent evolution, this is a comparison of positions in two mutant sequences originating from the same position of a common ancestor. Thus, the natural criterion for the matrix performance should be the effectiveness of the alignment procedure using this matrix.Existing amino acid substitution matrices can be divided into groups in accordance with the principles by which they were derived. The most representative group in terms of the number of matrices and applicability for the alignment procedure should include matrices of evolutionary nature, i.e. matrices obtained by comparing sequences. The most famous representatives of this group include matrices of universal application. These are the matrices series of PAM , Blosum The matrix constructed on the basis of the model of the Dirichlet mixture of the probabilities of the amino acid background  can alsoAnother branch in the group of evolutionary matrices consists of matrices aimed at comparing sequences of proteins belonging to a particular family , 11.Methods, Substitution matrices).This study was not limited to the evolutionary matrices specially developed for the alignment procedure; hence tests were also conducted on matrices created on the basis of alternative principles, using the alignment quality criterion , these substitutions are not fatal.Thus, it is of interest to know how universal the matrices are, what combinations of evolutionary distances of compared sequences and matrices are optimal, as well as comparisons of the efficiency of matrices obtained on the basis of sequences or three-dimensional structures.As a possible additional way of predicting matrix efficiency by evaluating a formal measure of the similarity of two matrices, we used the correlation coefficient. First of all, it was necessary to examine whether there is a relationship between the high correlation coefficient of the substitution matrices and the coincidence of alignment quality obtained using these matrices.Methods, Test sequences), and real sequences from Balibase [To evaluate the efficiency of the matrices, it is considered advisable to carry out the alignment of sequences of various origins. We used both model sequences constructed according to the scheme of divergent evolution  .In general, the alignment procedure is determined by the substitution matrix and the penalty function, which determines the contribution of insertion-deletions to the weight of an optimal alignment. In our study, we used a linear penalty function, which includes two parameters – a penalty for opening and continuing of insertion-deletion. We performed alignment with local  and globAccuracy are higher than Confidence at all points of the penalty function parameter area. Moreover, the difference between the characteristics of alignment quality for small values of the penalty function parameters is rather small, and for the highest values of the penalty parameters  this difference attains a significant value. This means that the number of comparisons in the reference alignment is greater than in the algorithmic local alignment for all values of penalty parameters. At the same time, for the global alignment the difference between Accuracy and Confidence values is significantly smaller, which indicates a negligible difference in the number of comparisons in the reference and in the algorithmic global alignments. Further, in all test sets for small values of the penalty function parameters, the Confidence values are greater than the Accuracy values, and for the largest values of the penalty parameters, this difference changes its sign.General tendencies of the quality changes in the alignment studied for the majority of the matrices, depending on the type of algorithm and the values of the penalty function, are presented in the example of PAM120 and Pfasum050 matrices in Figs. Accuracy, Confidence), the minimum was selected, then the maximum was selected from the set of obtained values. The values of the optimal alignment characteristics and the corresponding values of the penalty parameters for all the considered substitution matrices are shown in Tables The selection of an optimal value of penalty parameters was carried out as follows: of the two alignment characteristics . Finally, when aligned with the PAM250 matrix, on the contrary, with an increase in the evolutionary distance between sequences, the optimal gap opening penalty increases: GOP = 8, 11, 12.Other matrices constructed for large evolutionary distances behave similarly to the PAM250 matrix: Gonnet250: GOP = 8, 10, 11; Gonnet_p: GOP = 7, 8, 11; VTML250: GOP = 9, 11, 11; MIQS: GOP = 10, 12, 12.The remaining matrices give the worst values of accuracy and confidence; hence we did not discuss them.In this case, a simpler pattern was observed: with an increase in the evolutionary distance between sequences, the optimal penalty for gap opening increased (or does not decrease) for almost all tested matrices  than when matrices are used for a small evolutionary distance. For example, on the 30 PAM test set, the alignment quality using the PAM250, Gonnet250, and Gonnet_p matrices is slightly higher compared to the PAM60 matrix . The efficiency of the considered matrices of the Blosum series on the 120 PAM test set, as well as on Balibase sequences, is practically independent of the matrix number.For this type of alignment, in the case of remote sequences, the advantage of matrices for long distance over matrices for short distance is not as significant as in the case of local alignment. Therefore, on the 120 PAM test set  and a small value outside the diagonal elements (from − 15 to 2 for the matrix PAM30). This is because matrices for small evolutionary distances are designed to compare sequences containing a large proportion of matching characters. The alignment of sequences spaced a small evolutionary distance should have many symbol mappings and few deletion inserts, and to eliminate the occurrence of unnecessary insertion-deletions, a large penalty value is necessary (GOP about 19.0). At the same time, when aligned using both global and local algorithms, due to the large number of matches, maximum goal function is achieved over the entire length of the sequences (without dropping areas of low homology), and local alignment practically coincides with the global one. Since global alignment is the reference in our model for all evolutionary distances and types of algorithms, it gives a good quality of alignment.In the case of alignment of distant sequences with a local algorithm with the matrix for small evolutionary distances, a large value of the penalty which leads to the maximum goal function will be found, most likely on the fragment than on the whole sequence, which results in a mismatch with the reference global alignment.Accuracy < Confidence , which is clearly illustrated in Figs. The fact that, when aligning with a local algorithm with large values of the penalty function, alignments of the local type are obtained, leads to a much smaller number of comparisons in the algorithmic alignment compared to the standard alignment. This is equivalent to the inequality Accuracy = 0.165, Confidence = 0.283, see Table Accuracy < Confidence indicates a trend in the difference between algorithmic and reference alignments, expressed in a significant number of local type alignments in a set of algorithmic alignments.When considering the population of the aligned pairs, we can see that the low value of the average accuracy and reliability is the result of a large number of pairs with a complete mismatch with the reference alignment. When aligned using local algorithm, by reducing the penalty, as the calculation shows, it is possible to obtain the alignment of the global type, but with a large number of gaps and significantly different from the reference . With a large evolutionary distance between sequences (120 PAM), with penalties of GOP > 3, GEP > 2, the alignments obtained by the local algorithm have less similarity to the global alignment. However, the trend of increasing optimum value GOP with increasing evolutionary distances between sequences was observed.Matrices for a large evolutionary distance are characterized by a relatively smaller value of the diagonal and a larger value outside the diagonal elements (2..12 and − 8..7 for PAM250). The local alignment of sequences spaced by a small and medium evolutionary distance, with such matrices and gap opening penalty< 12, has enough similarities with global alignment, which explains the high similarity with reference alignment . The ranking of the considered matrices by the average value of two parameters of the alignment quality is given in Table Thus, we concluded that matrices corresponding to large evolutionary distances are universal. They not only align sequences best separated by large evolutionary distances, but align evolutionarily close sequences no worse than they align with matrices of the corresponding evolutionary distance. Calculations showed that among matrices corresponding to a large evolutionary distance, the Gonnet250 matrix gives the most stable alignment quality. The Gonnet_p matrix  obtainedThe use of simple methods for comparing matrices, such as calculating the correlation coefficients a priori, seems unpromising. Indeed, a comparison of the Table of correlation coefficients  show high correlation coefficients.r < 0.995), over the entire sample of matrices considered, despite the fact that the methods of their construction and initial data are different. It should be recalled that these same matrices showed the best alignment quality.Further, it is significant that large distance matrices of evolutionary type, such as PAM250, Gonnet, MIQS, VTML250, and Pfasum050, recorded the highest correlation coefficient between each other . Here, on the contrary, in the case of Johnson and Prlic matrices, the maximum correlation coefficient for the entire sample was achieved with matrices of evolutionary origin . The Blake matrix recorded the best correlation with the Johnson matrix. Thus, the correlation was partial, the generality of trends was not observed.Accuracy = 0.8011, Confidence = 0.7935), but it does not hold for the local algorithm for which the quality of alignment is noticeably lower  are also quite high. Nevertheless, it would be incorrect to identify the tendency of coincidence in the quality of alignment with a high correlation coefficient. The correlation coefficient score cannot compete with the alignment quality criterion.In this paper, we investigated the correspondence between the evolutionary distances of amino acid substitution matrices and the sequences to be aligned. It has been shown that, although at first glance one can limit the use of matrices with an evolutionary distance coinciding with the assumed evolutionary distance between sequences, a complete study of all combinations of matrices and sequences results in nontrivial conclusions. It was shown that the result of the alignment of sequences is separated by a greater evolutionary distance than the evolutionary distance of the substitution matrix. This shows a significant discrepancy between algorithmic and reference alignment.Conversely, in the case when the evolutionary distance between sequences is less than the evolutionary distance of the matrix, the coincidence of the obtained alignment with the reference is much higher.So for the main criterion of the quality of alignment, matrices corresponding to large evolutionary distances have a significant advantage. In contrast to the matrices for close distance, matching mainly equal coincident symbols in the alignment, matrices for long distance are able to match a. a. residues assembled by physical and chemical properties. Thus, not the literal coincidence of characters is preserved, but physical and chemical properties, of which hydrophobicity is the main one, as shown in .We examined the following question: Is there a relationship between the high correlation coefficient of the substitution matrices and the coincidence of alignment quality obtained with these matrices? It was noted that matrices corresponding to a high evolutionary distance, for which a high level of alignment quality is achieved, show a higher correlation coefficient between each other. Further, matrices corresponding to a small evolutionary distance, showing poor alignment quality, are characterized by lower correlation both between themselves and with matrices corresponding to a large evolutionary distance. Nevertheless, it would be incorrect to always identify the tendency of coincidence in alignment quality with a high correlation coefficient. Estimation by the correlation coefficient cannot compete with the criterion by the quality of alignment.The results obtained can serve as a recommendation for the practical use of the alignment procedure, especially in those cases when the evolutionary distance between sequences cannot be estimated.To evaluate the performance of substitution matrices for different evolutionary distances, the alignments of sequences of various origins were performed. We used both model sequences corresponding to different evolutionary distances and a sample of real pairs of sequences from Bali base , 19.S1 and S2 were generated from the original sequence S0 (“ancestor”). Thus, the process of generating a test pair consisted of two steps: generating a common ancestral sequence; generation of test sequences in accordance with the value of the PAM parameter  [The test sets of generated model sequences contained 1000 pairs of amino acid sequences in a 20-letter alphabet. Three sets were constructed using the same methodology, which differ in the values of evolutionary distance. Three sets were constructed using the same methodology, which differ in the values of the evolutionary distance. In the test pair, both sequences S0 was generated as a random Bernoulli sequence with a length of 200 a.a., comparable with the typical length of the polypeptide chain, with the frequency of amino acids occurrence in accordance with [The ancestral sequence nce with .S1 and S2 were constructed independently of one another. This procedure consisted of two stages. At the first stage, insertions and deletions were introduced into the ancestral sequence. For this, each position of the ancestral sequence S was checked for the occurrence of an insert before this position or deletion, starting from this position, with the probability of an event, according to [According to the same procedure, the sequences rding to :\documePAM is a number characterizing the evolutionary distance between the ancestral and mutant sequences.where The insertion or deletion length was randomly selected from the Zipf distribution, which, according to , does noPAM1 [id) and the value of the PAM parameter (according to Tables id) for two random sequences with an alphabet of length n and the same distribution of frequencies of characters occurrence  in the first and second sequences is defined as:At the second stage, point mutations were introduced into the sequence obtained at the first stage. Thus, mutations were introduced only in areas left over from the ancestral sequence. One cycle of introducing mutations was that in each position with a certain probability a substitution can be made, while the probability of a new symbol in this position is determined by the probability matrix  example ).id ≤ 35% according to [id ≤ 30% according to [For the distribution of amino acid frequencies from Dayhoff et al.  this valrding to , or 10% rding to .To determine the evolutionary distance between the modified sequences, on the basis of these alignments, the alignment of sequences S1 and S2 was constructed in such a way that positions originating from a common ancestor in sequence S0 were compared to each other.An example of text sequences shown in Fig. PAM (id) and PAM (indel)). This is due to the fact that with a sufficiently large number of accepted substitutions, the proportion of “visible” substitutions decreases.Table PAM with a decrease in id. Thus, in the resulting alignment, the number of “visible” substitutions corresponds to the sum of the substitutions made in the initial alignments, but does not equal the sum of the “visible” substitutions in them. At the same time, the total length of insertions and deletions in the resulting alignment, due to their small length relative to the total length of the sequences, is approximately equal to the sum of deletion inserts in the original alignments .In Fig. PAM parameter traditionally used to characterize the evolutionary distance is oriented to the second scheme. Thus, the actual PAM value for the second circuit is approximately two times greater than the specified value. This evolutionary model was used by the authors in [Note that in the described procedure, test pairs of sequences were built according to the “descendant1 - descendant2” scheme, and not according to the “ancestor-descendant” scheme. The first scheme better models the comparison of real sequences. However, the thors in  to generthors in , 26.Accuracy and Confidence measures described in [We were interested in how close algorithmic alignments are to reference alignments, i.e. alignments in which positions originating from the same position of the ancestral sequence are matched. To assess the degree of this proximity , we used the ribed in , 25, 27:Accuracy = I/R, Confidence = I/A, where I is the number of matching comparisons in the reference and algorithmic alignments, R is the total number of comparisons in the reference alignment, A is the total number of comparisons in algorithmic alignment.The alignment of all sequences was performed by two algorithms: the local Smith – Waterman algorithm , and theWe examined matrices of various types and origin in order to evaluate their performance in restoring the reference alignment, i.e. one in which positions originating from the same position in a common ancestral sequence are matched. Table Further, the matrices considered are described in more detail.Point accepted mutations matrix series became the first kind of evolutionary matrix. In order to obtain it, Dayhoff et al. estimated amino acid substitutions in 72 groups of closely related proteins (id> 85%), consisting of more than 1300 sequences. PAM1 matrix was obtained as a result of the calculation of amino acid residues substitutions frequencies. It corresponds to an evolutionary distance unit of 1PAM, at which 1% of residues are substituted. The remaining PAM matrices were obtained from the original PAM1 matrix by raising it to the power, which is the value of the evolutionary distance. The remaining PAM matrices can be obtained from the original PAM1 matrix due to its exponentiation, which is also an evolutionary distance value. Thus, matrices with a larger number correspond to a larger evolutionary distance. Note that with an increase in the number of the PAM matrix, the relative value of the diagonal terms decreases and approaches the average value of the off-diagonal terms. Further, from the substitution frequency matrices, the substitution weight matrices were obtained, whose elements are the logarithm of the ratio of the substitution frequency to the product of the frequencies of occurrence of the corresponding amino acids.Blosum series matrices were developed on the basis of multiple comparison  of relative motifs segments contained into blocks. Single block presents conservative protein series zone as relative segments of individual proteins, which are located one below the other. In other words, this single block presents a two-dimensional array, where each line is a protein sequence segment, and each column shows the position of leveled balance.In total, about 2000 blocks of aligned sequence segments were defined characterizing more than 500 groups of related proteins. Then these blocks were combined into groups in accordance with the identity of their segments. To decrease the dependence on amino acid matching frequency, these sequences were grouped in the clusters inside blocks. Each cluster was considered as one sequence, when matching of pairs was calculated. Based on the substitutions frequency in each group, its own Blosum matrix was built. A matrix number means the amount (percentage) of identities, which is typical for the group.Thus, unlike PAM matrices, Blosum series matrices are obtained on the basis of direct data, and not by extrapolation from a small evolutionary distance to a large one. In this case, a larger number corresponds to a smaller evolutionary distance.Gonnet250 matrix was developed with almost the same methods as PAM series matrices, but significantly larger data sets were used. This provides it with better validity compared to PAM matrices. Therefore, the total length of the sequences from the database was more than 8 million amino acid residues. Each sequence was compared with the entire database, as a result of which 1.7 million alignments were obtained. On this basis, a matrix was constructed corresponding to the evolutionary distance of 250 PAM.The matrix was obtained from the Gonnet250 matrix by increasing all its elements by a constant value to get positive values for all elements of the resulting matrix.This evolutionary matrix can be interpreted as the matrix of the “third generation”. To create a new matrix intended to remote homologues recognition, a test set based on the Cluster of Orthologs Groups (COG)  was builThe matrices of the VTML series are a later development of the model originally implemented in the matrices of the PAM series  and thenThis matrix is based on information associated with existing matrices condensed into a new matrix that can detect more distant homologues. For this purpose, matrices of Blosum, and VTML series; designed by Benner et al.  were selMatrices of this series were obtained from multiple alignments of the sequences of seeds of the families of the Pfam protein database (version 29.0) , which aThus, Pfasum matrices are based on modern data, covering a large and diverse sequence space of 47.3 billion amino acid pairs in 16,295 multiple alignments. In addition, most existing substitution matrices are derived only from highly conservative or filtered sequence data by excluding regions containing gaps or ambiguous amino acids. By contrast, the PFASUM matrices design takes into account all information.This research was inspired by the fact that the dynamics of substitutions in amino acid sequences at large and small evolutionary distances are different. Previously, this matter was considered in , 5, 16. Plasmodium. The use of these matrices reduces the number of false positive hits when searching for homologues.The aim of this study was to improve the search for homologues in families of related proteins, taking into account the bias of the amino acid composition characteristic for proteins of specific species. For this purpose, a substitution model based on “fuzzy” clustering is proposed, in contrast to the hierarchical clustering used in the construction of the Blosum series matrices. To calculate the matrices, 1834 multiple alignments were used from the BLOCKS database, corresponding to the amino acid composition characteristic of the proteins of two species of the genus Mollicutes class were used, with genome biased towards A + T. Of the 14 bacterial genomes of the Mycoplasma species, by finding the greatest similarity , 247 orthologous protein families were selected. To construct the matrix, the approach described in [This paper presents a general method for constructing matrices focused on a certain bias of amino acid composition. As initial data, proteins of bacteria belonging to the This matrix is based on the calculation of substitutions of amino acid residues in the structural alignments of 235 proteins from 65 homologous families. Most of the data used have a relatively low sequence identity of 15 to 40%.Matrices were obtained on the basis of superposition of pairs of proteins having a similar structure, but low sequence identity, and are intended to compare evolutionarily distant proteins. Since structural alignments have several solutions, especially with low sequence identity, the following question arises: which of them is the most believable in terms of evolution? To build the matrices, alignments with the largest number of matching residues were used. Matrices obtained both on the basis of only homologous pairs and the entire set of structural alignments, including homologues and analogues, were considered.The study is based on two observations: data on structural superposition give a training sample to improve the alignment of distant homologs; and the main substitutable arrays of amino acid residues for distant homologs differ from those for closely related proteins. Based on the data of structural superpositions, a set of matrices of amino acid residue substitutions was constructed. These matrices use a known structural homology as the characteristics of the impact of evolution on the residue substitution profiles characterizing multiple alignment.The matrix was calculated on the assumption that for protein sequences spaced 1 PAM distance, the genetic code is the only factor leading to a drift in the amino acid composition. In other words, the influence of the code  can occur either at the smallest evolutionary distances or in parts of the tertiary structure that have no effect on the function of the protein. To build the initial matrix, a database was used, including 1.7 million pairs of sequences. Further, the resulting matrix was extrapolated to a distance of 250 PAM by raising to the appropriate degree.The matrix is based on an estimation of the distribution of contacts in three-dimensional protein structures. The coordination number per residue is optimized in order to find the best fit between the observed and predicted partial energy. These new contact energies make it possible to improve their ability to discriminate the native structure from non-native folds in the dragging procedure.Additional file 1: Table S1. Local alignment. Test sets: 30 PAM, 60 PAM, 120 PAM, Balibase.Additional file 2: Table S2. Global alignment. Test sets: 30 PAM, 60 PAM, 120 PAM, Balibase.Additional file 3: Table S3. Correlation of matrices.


Advances in sequencing technology have made large amounts of biological data available. Evolutionary analysis of data such as DNA sequences is highly important in biological studies. As alignment methods are ineffective for analyzing large-scale data due to their inherently high costs, alignment-free methods have recently attracted attention in the field of bioinformatics. In this paper, we introduce a new positional correlation natural vector (PCNV) method that involves converting a DNA sequence into an 18-dimensional numerical feature vector. Using frequency and position correlation to represent the nucleotide distribution, it is possible to obtain a PCNV for a DNA sequence. This new numerical vector design uses six suitable features to characterize the correlation among nucleotide positions in sequences. PCNV is also very easy to compute and can be used for rapid genome comparison. To test our novel method, we performed phylogenetic analysis with several viral and bacterial genome datasets with PCNV. For comparison, an alignment-based method, Bayesian inference, and two alignment-free methods, feature frequency profile and natural vector, were performed using the same datasets. We found that the PCNV technique is fast and accurate when used for phylogenetic analysis and classification of viruses and bacteria. Predicting the structures, functions, and evolutionary relationships of genes is a fundamental and vital aspect of modern biological research. Therefore, the comparison of genetic sequences is a pivotal step in many protocols and numerous approaches have been employed for this task. Most researchers use conventional alignment-based techniques for sequence comparison; these techniques involve sequence alignment based on selected scoring systems. The algorithms used are generally precise and highlight correlations among sequences. Several sequence alignment methods have been implemented via software packages, such as MrBayes . HoweverAlthough k-mer-based methods have been applied widely, they do not include positional correlations of nucleotides. However, it is significant to investigate the location for gene sequence comparison. Therefore, positional correlation is important for computational and analytical approaches. Recently, two methods based on moments of the positions of the nucleotides, namely natural vector (NV)  and multTo demonstrate that PCNV is effective, we applied it to different datasets: the genomes of hepatitis C virus (HCV), hepatitis B virus (HBV), human papillomavirus (HPV), dengue virus (DENV), and 59 bacterial species. The length of the sequences studied ranged from thousands to millions of base pairs. For each dataset, the PCNVs of the sequences were computed using MATLAB R2016a and phylogenetic trees were reconstructed using MEGA 7. Finally, we evaluated the performance of our methods based on sensitivity, specificity, and accuracy. Computations were performed on a PC with Intel Core i7-6560U CPU @ 2.20 GHz and 8 GB RAM.Using our PCNV method, 82 HCVs are correctly clustered into six clades, as shown in Using PCNV, 152 HBVs are correctly divided into eight lineages, as shown in As shown in We found that PCNV categorizes the dataset into the correct biological groups in 0.78 s ; this isThe dataset consisted of 14 families, as shown in Besides evolutionary analysis, PCNV can also be used for classification. Both FFP and Alignment-Free-Kmer-Statistics (AFKS) , based o studied . For thehe set S .In the PCNV method, after computing the distance matrix using each approach, the one-nearest neighbor (1-NN)  method wIn the present paper, we propose a novel 18-dimensional vector method for genome comparison. This PCNV method can be used to successfully define the distribution of nucleotides based on information on the frequency and position of DNA sequences. The correlation of position between two different bases is used in addition to the average position and variance of position of each base. As a result, a high-dimensional DNA genome sequence is reduced to an 18-dimensional numerical vector. Correlations in base distribution play a key role in sequence comparison. Usually, conventional alignment-based methods produce reasonable phylogenetic trees and are therefore widely applied. However, when the dataset volume is large or the sequences analyzed are very long, these methods become ineffective. The phylogenetic analysis results on several distinct datasets show that PCNV can quickly and accurately compare massive datasets of long DNA sequences. We also compared our method with three methods: the popular alignment-based Bayesian inference method, the alignment-free FFP, and Natural Vector methods. The results show that our method can construct more accurate evolutionary relationships among sequences.To demonstrate the computational advantage of PCNV, we compared our running time constructing phylogenetic trees with FFP, AFKS, Bayesian inference, and Muscle. Compared with the two extensively applied alignment-free methods FFP and AFKS, the running time of PCNV is smallest for all datasets and even takes less than 1 s, except on the bacteria dataset. For bacteria dataset, PCNV is extremely fast and only takes about 53 s, while FFP and AFKS take more than one day. Compared with the two alignment-based methods Bayesian inference and Muscle, PCNV takes much less time for all datasets. For bacteria dataset, Bayesian inference and Muscle cannot obtain phylogeny tree within several days.With long DNA sequences, particularly bacterial genomes, the Bayesian inference method was much slower than the PCNV approach, sometimes to the extent that it simply did not work, the main reason being that Bayesian inference method is based on alignment method, such as Muscle . Even alFurthermore, the MEV was used for comparison. Although both PCNV and MEDV studies aim to solve the problem of genetic sequence comparison, they have significant differences in the features extracted from the sequences. MEV method did not consider the position correlation feature which is an important source of information for genetic sequences. The novelty of our new method is that it designs six suitable features to characterize the correlation among nucleotide position in sequences. The second difference is that our new method does not categorize four types of nucleotides into three groups according to their chemical properties. The third difference is that our new method applies the popular neighbor-joining algorithm to construct phylogenetic trees, while the old study used the UPGMA algorithm which may produce misleading trees.To show the advantages of our new method (PCNV), MEV and PCNV were compared using all datasets studied used in for the present study. The Neighbor-Joining (NJ) algorithm is used in tree construction of both methods. The NJ trees built by the MEV method are shown in Horizontal  gene transfer (HGT) is a common phenomenon in bacteria. Due to the problem of HGT, Koski and Golding  even staYersinia pseudotuberculosis and six Yersinia pestis complete genomes. Using PCNV, we get the neighbor-joining tree of the eight bacteria shown in Yersinia pseudotuberculosis isolates form sisters and are separate from the six Yersinia pestis genomes.To show that our method may be not affected by HGT, we used another dataset of eight Yersinia genomes in a previous study . The eigGenetic distance is a measure of the genetic divergence between species or between populations within a species, whether the distance measures time from common ancestor or degree of differentiation. Several genetic distances have been proposed based on different evolutionary models. The genetic distance can only be applied to alignment results. A commonly used measure of genetic distance is the fixation index, which varies between 0 and 1. Our PCNA is an alignment-free approach; we measure the distance between species using Euclidean distance and we do not assume any evolutionary model. This distance is positively correlated with their genetic distance, since it can successfully measure the divergence between two species as well. In the neighbor-joining trees constructed by PCNV, sum of the length of the branches traversed from one species to another is equal to their distance and thus positively correlated with their genetic distance. Due to difference in mutation rates for species, for two given datasets, the average internal genetic distance in each dataset may be different enough. In this case, the average distance obtained with PCNV for two datasets may have a big difference, which leads to very different scales in the two derived phylogenetic trees. For the same dataset, different alignment-free method may produce different average distance and thus produce different scales of phylogenetic trees.Our new method has several limitations, for example we cannot work out the time of evolution. These limitations will need to be studied in the future.Five datasets were used to test and verify the new technique.Hepatitis C is a liver infection caused by HCV. The World Health Organization (WHO) estimates that HCV infects 3% of the world’s population . BecauseHBV is a hepatotropic virus that can establish a persistent and chronic infection in humans through immune anergy. It exhibits formidable morbidity and mortality in humans and currently infects 3.5% of the global population ,29. HBV DENVs are mosquito-borne aviviruses that have plagued humans for centuries . StatistHPV is a circular double-stranded DNA virus that causes a variety of proliferative epithelial lesions at specific anatomical sites. It is also the most common sexually transmitted virus. There are many different types of HPV, several of which cause health problems such as genital warts and cancer . For exaThere is a biomass of bacteria, the main representatives of the prokaryotes, on Earth. Researchers usually investigate evolutionary relationships among bacteria by building phylogenetic trees. Owing to the large genome size (>1 million bp) of bacteria, it is impossible to reconstruct a bacterial phylogenetic tree in a reasonable amount of time using traditional multiple sequence alignment methods with current computational technology. We used 59 bacterial species to test our method . The NCBN. Denote jth appearance, Let The positional distribution of Example: Given the sequence “ACTGGCAAT”, the positional distribution is as shown in (1)When (2)When (3)When Similarly, we can get The average positional distribution Therefore,For the example sequence “ACTGGCAAT” above, The positional covariance of nucleotide For the sequence “ACTGGCAAT”, the positional distributions of A and C are The same method is used to compute cov ; cov ; cov ; cov ; and cov .The positional variance Consequently, an 18-dimensional PCNV of a DNA sequence was constructed as follows:Then, we used the Euclidean distance to compute the pairwise distance among the 18-dimensional vectors of the genome sequences. A phylogenetic tree can be built using the NJ algorithm in MEGA 7.0 software .Additionally, the PCNV method can be used to classify organisms. Naturally, sensitivity, specificity, and accuracy were used to evaluate classification performance. The definitions of these measures are as follows:


The dynamic time warping (DTW) algorithm is widely used in pattern matching and sequence alignment tasks, including speech recognition and time series clustering. However, DTW algorithms perform poorly when aligning sequences of uneven sampling frequencies. This makes it difficult to apply DTW to practical problems, such as aligning signals that are recorded simultaneously by sensors with different, uneven, and dynamic sampling frequencies. As multi-modal sensing technologies become increasingly popular, it is necessary to develop methods for high quality alignment of such signals. Here we propose a DTW algorithm called EventDTW which uses information propagated from defined events as basis for path matching and hence sequence alignment. We have developed two metrics, the error rate (ER) and the singularity score (SS), to define and evaluate alignment quality and to enable comparison of performance across DTW algorithms. We demonstrate the utility of these metrics on 84 publicly-available signals in addition to our own multi-modal biomedical signals. EventDTW outperformed existing DTW algorithms for optimal alignment of signals with different sampling frequencies in 37% of artificial signal alignment tasks and 76% of real-world signal alignment tasks. Dynamic time warping (DTW) is a class of pattern matching algorithms for aligning temporal sequences. DTW aims to provide an optimal similarity-based match between two sequences regardless of dynamic spatiotemporal differences, searching for the optimal path by minimizing cumulative point-to-point distances. DTW allows for a flexible definition of cumulative point-to-point distances to optimize alignment performance for different datasets. Because of this flexibility, DTW is widely used in speech recognition under varying speaking speeds , gestureMeasurement modalities utilizing multiple simultaneous sensors with different sampling frequencies and/or uneven sampling rates are common in many applications, particularly in biomedicine ,6. In prA real-world example of a singularity problem using two biomedical devices to measure the same biological phenomenon (heart rate) with two different modalities with different sampling frequencies  and photoplethysmogram (PPG), blue (~0.2 Hz)) is shown in Various improved DTW algorithms have been developed and applied to different non-temporal datasets ,10. KeogA common method employed to reduce instances of singularities involves adding penalties to the cumulative distance minimization function in the DTW algorithm . Early aDespite extensive research on DTW algorithm development to solve the singularity problem, methods to evaluate signal alignment are currently limited. Since DTW is a matching result calculated from distance measurements but not a metric , we needFurther, the singularity problem in DTW continues to be a challenge when aligning multi-modal signals. Here we propose an improved DTW algorithm, EventDTW (eDTW). eDTW takes local structural information into account when aligning signals. Although there have been different attempts to align signals with dynamic time warping ,19, theyhttps://github.com/Big-Ideas-Lab/DBDP/tree/master/DigitalBiomarkers-Preprocessing.) To enable comparison between algorithms, we designed two metrics, the singularity score and the error rate, to evaluate these two aspects, respectively. Using a gold-standard open-access public dataset [In this paper, we propose an improved DTW algorithm and evaluate its performance against existing DTW algorithms in terms of the severity of the singularity problem and the accuracy of the resulting alignment between signals.  as a metric to quantify the quality of an alignment between two signals, i.e., how different any given alignment is from the optimal alignment. We defined We quantified misalignment of each observation in In order to measure the severity of the singularity problem, we proposed the singularity score (SS) by introducing a ratio inspired by the variance calculation for a dataset:In comparison with the warping path metric ing path , the squHowever, the singularity problem was directional when the frequencies of signals differed. This meant that the situation where EventDTW was designed to improve the alignment of two signals when they measure the same phenomenon with different frequencies. Considering that global signal shape- and derivative- based algorithms like sDTW and dDTW ignore key local information, we propose a DTW algorithm that takes advantage of local information to produce better overall alignments with respect to the whole signal. Inspired by the design of the Hawkes process , we intrSelect an entire upslope with monotonically increasing observations in the signal. An upslope is defined such that it is a contiguous subset of observations starting with the lowest point and continuously increasing until there is a decrease or stagnation from one point to another.Select the key slopes. We selected the slopes with the highest 20% elevation in Every upslope in Set the peak of the upslope as an event and propagate the information along the slope as per below.Since the trends of a signal segment can be classified as either an upslope, a downslope, or a plateau, by our definition we denoted the types of events as We denoted the The decay process of the event information was exponential. We set The event information propagation of Both slope detection and key slope selection required looping over values of For scaling EventDTW, several components for the methodology could be optimized. For the slope calculation, event matching, and event information propagation, parallelization could be applied for scaling since the individual computations were not dependent. However, for the key slope selection step, synchronization was needed because the key slopes were selected based on the comparison of all detected slopes, and this did not enable parallelization. The standard DTW method could be optimized to mine massive datasets, and since EventDTW is calculated using standard DTW with the inclusion of event information, EventDTW is similarly scalable .https://www.cs.ucr.edu/~eamonn/time_series_data/) by introducing a localized time shift and an amplitude distortion to the signal amplitudes. Perfect alignment of this warped time sequence to the original, optimal sequence would give an exact one-to-one index correspondence, i.e., a 0% error rate. The error rate compares the alignment resulting from using a DTW algorithm against a perfect alignment that we calculated in II.B, and outputs a proportion score of error.In order to compare the performance of eDTW against existing state-of-the-art DTW algorithms, we synthetically warped a set of 84 publicly available time series datasets  and photoplethysmogram (PPG) sensing using wrist and chest measurement modalities. ECG is a test that measures the electrical activity of the heart. PPG is an optically obtained plethysmogram that can be used to detect blood volume changes in the microvascular bed of tissue. In this study, we experimented with a dataset that we collected  consistiWe calculated heart rate (HR) from the ECG using the standard RR intervals, which is the time elapsed between two successive R-waves of the electrocardiogram, using Kubios HRV Premium (version 3.3). Heart rate values in units of beats per minute at each timestamp were calculated by taking the inverse of the RR interval and multiplying by 60. We compared methods to align the processed heart rate signals from the ECG and the Apple Watch 4. We preprocessed both the ECG and PPG heart rate data using Gaussian smoothing kernels and standardized the signal amplitude. We then aligned the PPG and ECG signals using four different algorithms, DTW, dDTW, sDTW, and eDTW, and calculated the SS for each.In the experiments applying DTW algorithms on the biomedical dataset STEP , we obseThe evaluation of DTW algorithms has proven to be a significant challenge. With the two evaluation metrics proposed here, we have enabled for the first time a quantitative method to evaluate and compare DTW algorithm performance. Further, our results demonstrate improved signal alignment using our novel algorithm, EventDTW, as compared to the original DTW algorithm for a significant number (36.9%) of cases in the 84 gold-standard signal processing datasets. EventDTW is also strictly superior to the dDTW and sDTW algorithms in addressing the singularity problem when the signals to be aligned are of different frequencies. However, as can be seen in We believe that eDTW has the potential to significantly outperform other DTW algorithms in specific circumstances, one of which is when the two signals we attempt to align are measured at different frequencies and have distinguishable events. However, signals of different characteristics may require different DTW algorithms for optimal alignment. The more pronounced the events in the signals are, the likelier eDTW is to find the global optimum alignment. We believe that eDTW is flexible and can be adjusted to different situations and may produce the optimal alignment under nearly all circumstances if the choice of the events is carefully delineated prior to performing alignment with eDTW. Future work will involve designing and testing a signal characteristic metric to determine the optimal DTW algorithm, or algorithm parameters will optimize alignment for signals of different characteristics.In this article, we compared existing DTW algorithms and illustrate their limitations with regard to the singularity problem and accuracy in aligning signals of different sampling frequencies. We detailed the construction of the error rate and the singularity score metrics for evaluating and comparing DTW algorithm performance. We then demonstrated the significant potential of EventDTW in addressing singularity problems and producing accurate alignments by applying eDTW to both synthetic time sequences as well as real-world biomedical signals. While several limitations still exist and future steps of research and analysis are needed, the eDTW algorithm outperforms existing DTW algorithms for aligning real-world biomedical signals and addresses a wide need in this application area.


The constant growth of genomic data intensifies development of data structures and algorithms to investigate pan-genomes efficiently.The term affinity tree – is proposed. Each node of this tree has assigned a subset of genomes, as well as their homogeneity level and averaged consensus sequence. Moreover, subsets assigned to sibling nodes form a partition of the genomes assigned to their parent.This work focuses on providing a tool for discovering and visualizing the relationships between the sequences constituting a pan-genome. A new structure to represent such relationships – called Ebola virus pan-genome. Furthermore, two software packages are provided: PangTreeBuild constructs affinity tree, while PangTreeVis presents its result.Functionality of affinity tree is demonstrated on simulated data and on the  The amount of genomic data is enormous and its growth rate is increasing constantly. As an example, bacteria genomes availability has risen over 10,000-fold over last 25 years [pan-genome has emerged. Initially proposed as a single data structure for joint analysis of a group of bacterial genes [Consequently, the idea of a al genes , in the al genes .Simultaneous access to many genomes makes the data structure information-rich. Pan-genome can serve as a demonstration of single organism’s genome compared to its relatives, but also as a complex scheme exposing the variety of the component sequences. The first perspective is particularly important for applications in personalized medicine, because it makes possible to highlight one’s individual properties which can be crucial in the mean of its phenotype. The other one can be useful when a reference genome for a population is needed.Several pan-genome models were proposed, ranging from collections of unaligned sequences to sophisticated (e.g. HMM-based) models that require complex preprocessing of sequence data . Compromblocks, which represent aligned fragments of input sequences as classical colinear MSA. A small example of such block alignment scheme is presented in Fig. The classical MSA model has a form of a matrix with rows obtained from input sequences by inserting gaps in appropriate positions. This model has several advantages, e.g. matrix columns naturally define a joint coordinate system for all input sequences. Additionally, most common sequence evolutionary events – insertions, deletions and substitutions – are well highlighted. However, such representation cannot handle rearrangements that violate colinearity of input sequences, e.g. duplications or inversions. In order to properly represent such rearrangements, the whole alignment is usually split into Sequence graphs naturally represent rearrangements of any kind. Moreover, in these models identical fragments of different input sequences may be combined into one, reducing the MSA redundancy in sequence representation. Such reduction is particularly important when a pan-genome is used for read mapping. However, it comes with the cost of loosing information regarding the input sequences, so additional graph annotation is required to provide a common coordinate system.Partial Order Alignment (POA) [Graph-based MSA models were also proposed. One of the first such solutions was probably nt (POA) . In thisSeveral graph-based alignment models were applied to the problem of whole genome alignment, e.g. A-Bruijn graph , Enredo sequence profile is a vector of symbol frequency distributions in particular MSA columns, consensus sequence is composed of symbols dominating consecutive columns, while information content vector quantifies the levels of that domination.All multiple sequence alignment representations are complex, even classical colinear one. Some MSA applications require previous extraction of selected properties that could be represented in a more compact way. For example, The above concepts could be straightforwardly extended to most graph-based alignment models, but we anticipate possible results unsatisfactory for at least two reasons. Firstly, the result would be still a sequence graph with the same or similarly complex structure, and this structure is what makes the graph alignment hard to use. Secondly, in all the above objects MSA columns are treated independently, while graphs encode some dependency between residues . We find this property an important advantage of graph-based alignment models and expect it to be reflected in the result.One of the possible solutions was proposed in : insteadThe concept of POA consensus sequences has found several applications. Examples include paralog separation in EST alignments , gene ispan-genome may be split into different genome subsets, depending on the assumed subset homogeneity level,subsets of a wide range of levels can contribute to the description of the relationships between genomes; the same applies to their consensus sequences,the image of the pan-genome structure should be complemented by subset hierarchy.The aim of this article is to fill this gap. For this purpose we extend the above concept, based on the following principles:affinity tree that meets the above requirements. Furthermore, we propose an algorithm computing affinity tree for given MSA, implemented in the package PangTreeBuild. Functionality of our approach is evaluated on simulated data and on the MSA of Ebola virus genomes. Finally, we present a tool for affinity tree visualization, called PangTreeVis.In the following sections we introduce the notion of Partial Order Alignment (POA) [nt (POA)  was probHeaviestBundle algorithm searches the heaviest path in the graph using dynamic programming. Next, the algorithm calculates the compatibility of each sequence with the consensus, defined asNi is the length of seuquence i and Nij is the number of POA nodes belonging to both the sequence i and the consensus j. The sequence is assigned to the consensus, if the result is larger than the assumed threshold. Finally, edge weights are recalculated and the whole procedure is repeated for remaining sequences until all sequences are assigned or none of them exceeds a compatibility threshold. This forms a partition of the input sequences into subsets with assigned consensus paths.Lee  presenteaffinity tree allows to represent the hierarchy of sequence subsets. Each node of this tree has the following attributes assigned:a subset of input sequences,a consensus sequence being their common representation,minComp).minimum of the sequence-consensus compatibilities  creating a node covering all remaining sequences. The second modification restricts searching to compatibility gaps overlapping the interval covering the minComp values of a parent and already created siblings . Although the restriction could potentially cause choosing an accidental threshold, the interplay between data-driven threshold selection and iterative consensus recalculation (see Algorithm 2 for details) minimizes this effect.Since the parent P that controls the granularity level of the resulting tree. For the purpose of calculation of the largest gap compatibilities and minComp values are transformed according to the formulaP>1 the distance between smaller compatibilities decreases and the distance between higher compatibilities increases. Consequently, the sequence set assigned to a particular node is split into smaller and more homogeneous subsets and hence more children are created for the node. The opposite happens for P<1. The effect of this parameter is presented in Fig. Finally, we introduced an additional parameter The algorithm was implemented in software called PangTreeBuild, which is available as a Python 3.6 package or a console application for Linux operating system. Moreover, it could be executed from the PangTreeVis web browser interface see Fig. .Fig. 5UMafgraph tool [As an input PangTreeBuild requires a block alignment in MAF format, which is widely used to represent whole genome alignments and can be interpreted as a pan-genome model. PangTreeBuild internal data model is POA so alignment blocks are converted to it as shown in Fig. aph tool . Next, uP and stop, AffinityTree procedure is run. The constructed data model together with the resulting affinity tree are saved as a JSON file, which can be visualized using PangTreeVis tool.Given such a pan-genome model and parameters PangTreeVis is an interactive browser application. The visualization has two parts see Fig. . The firEbolavirus genomes).The PangTreeBuild algorithm was applied to two types of MSA datasets: simulated (yielded from genome sequence evolution simulations) and real-life . In the second one, genome rearrangement events were added: intra-chromosomal inversions, tandem repeats, transpositions and duplications, as well as inter-chromosomal transpositions and duplications.The root sequence consisted of two 100The simulations were guided by a phylogenetic tree of 138 yeast strains  metric, we added to each leaf children, representing original sequences assigned to a corresponding subset. Results are presented on Fig. For each MSA affinity trees were generated by PangTreeBuild with parameters stop =0.999 and % of the distances for PangTree affinity trees and >95% of the distances for Lee’s affinity trees. However, in the last case this is very close to the total number of non-trivial splits in the phylogenetic tree (i.e. 137), which means that only very few of them are shared by Lee’s affinity trees.RF distance is defined as the number of splits that are induced by one of the trees but not by the other. The number of splits induced by a single tree equals the number of its nodes. For a fixed number of leafs, the largest possible number of splits is obtained by bifurcating trees. In our case phylogenetic tree is bifurcating, while affinity trees are highly multifurcating, since they are intended to express only the most evident partitions. Consequently, the part of the RF distance that is due to splits induced by the phylogenetic tree only reflects the granularity of affinity trees rather than the inconsistency between trees. As is shown on Fig. The above observation is confirmed by Fig. The red bars on Fig. minComp as a subset homogeneity measure, we compared it against the corresponding measure in the phylogenetic tree – maximal distance from an internal node to its leafs. Results are presented on Fig. In order to evaluate the behavior of ebolavirus and 2 genomes of marburgvirus (the closest Ebola relative). Every sequence from this alignment is assigned to one of the seven groups: Ebola 2014, Bundibugyo 2007, DRC 2007, Reston 1989-90, Sudan 1976, Zaire(DRC) 1967-7 and Marburg 1987. This division slightly differs from official taxonomy [Ebola outbreak in 2014 intensified studies concerning this dangerous virus. As it is crucial to find cure and vaccine for the illness it causes, a large set of sequenced samples was prepared. The data and associated studies are collected in UCSC Ebola Portal . The datkbp. Due to high sequence similarity POA for all 160 genomes contains only 70603 nodes. Affinity tree computed with the parameters P=0.25, stop =0.99 is presented in Fig. The length of a single genome of Ebola or Marburg virus is ∼19ebolavirus genomes from marburgvirus genomes. Ebolavirus genomes are later split into 5 subsets, corresponding to 5 ebolavirus species represented in the dataset: Tai Forest, Bundibugyo, Reston, Sudan and Zaire. The subsets differ in homogeneity: minComp ranges from ∼0.96 for Sudan and Restonebolavirus to ∼0.99 for Zaire ebolavirus.As we can see, the only non-leaf child of the root separates all Zaire ebolavirus and other species, as well as those between Tai Forest and Bundibugyo ebolaviruses.In order to look into details of the relationships between species, we calculated compatibilities between their consensus sequences. Results are presented in Table local compatibilities between consensuses of the ebolavirus species. Results presented on the Fig. kbp), is known to exhibit some interspecies variation [The compatibility measure describes similarity averaged over the whole sequence. In order to analyze its variation over the genome sequence, we calculated ariation .Fig. 11Computational requirements of affinity tree calculations for datasets used in this study are compared in Table hierarchy of most evident homogeneous sequence subgroups,subgroup reference (or consensus) sequences,graph-based multiple sequence alignment, joining input and reference sequences,local and global sequence homogeneity measures.The relationships between homologous sequences are usually represented using two complementary structures – phylogenetic tree and multiple sequence alignment. In the current work we proposed a novel structure, called affinity tree, which joins both perspectives. As opposite to phylogenetic trees, affinity trees are not intended to be a detailed reconstruction of evolutionary history. Instead, they provide:Consequently, affinity tree can serve as a pan-genome model, supporting interesting features for comparative genomics studies. Given a whole-genome alignment, affinity tree specifies homogeneous subgroups of contributing genomes. Each subgroup is characterized in terms of its consensus sequence and subgroup genetic diversity. Graph-based alignment induced on consensus sequences represents spatial distribution of similarity between subgroups, while alignment between sequences constituting a subgroup and their consensus – of similarity within the subgroup. Furthermore, spatial distribution of compatibility of mixed individual’s genome with consensus sequences of various subgroups can delineate the mosaic structure of that genome. In summary, graph-based alignment of consensus sequences can serve as a diversified population reference genome.Several practical issues should be considered when using affinity trees in comparative genomics studies. First, joint visualization of the features supplied by the affinity tree model is challenging. PangTreeVis provides the visualization of the alignment graph  and of the tree structure, enables interactive subgroup selection and viewing genome-subgroup relationships. Although the basic requirements are satisfied, the advantages of the affinity tree model could be highlighted better if some extensions were provided, e.g. graph visualization on intermediate scale levels, support for local genetic diversity layer and additional layers .Second, the quality of the affinity tree hardly depends on the provided whole-genome alignment. PangTreeBuild algorithm was designed to pan-genomes with limited number of structural variants. It performs well on datasets examined in the current study, but our experiments show that large-scale rearrangements influence both the algorithm efficiency and the quality of resulting affinity tree. Consequently, complex alignment graphs may require an adjusted building algorithm.Affinity tree and its generation algorithm support the pan-genomic research field. Apart from hierarchical division of aligned genomes, affinity tree describes homogeneity of resulting subgroups and provides subgroup reference sequences. The introduced method gives new insight into multiple sequence alignment analysis – its result can serve as both taxonomic study and a population reference pan-genome. Two complementary software packages: PangTreeBuild and PangTreeVis enable affinity tree construction and visualization.