The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Abstract ::: Techniques to determine changing system complexity from data are evaluated. Convergence of a frequently used correlation dimension algorithm to a finite value does not necessarily imply an underlying deterministic model or chaos. Analysis of a recently developed family of formulas and statistics, approximate entropy (ApEn), suggests that ApEn can classify complex systems, given at least 1000 data values in diverse settings that include both deterministic chaotic and stochastic processes. The capability to discern changing complexity from such a relatively small amount of data holds promise for applications of ApEn in a variety of contexts. <s> BIB001 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Entropy, as it relates to dynamical systems, is the rate of information production. Methods for estimation of the entropy of a system represented by a time series are not, however, well suited to analysis of the short and noisy data sets encountered in cardiovascular and other biological studies. Pincus introduced approximate entropy (ApEn), a set of measures of system complexity closely related to entropy, which is easily applied to clinical cardiovascular and other time series. ApEn statistics, however, lead to inconsistent results. We have developed a new and related complexity measure, sample entropy (SampEn), and have compared ApEn and SampEn by using them to analyze sets of random numbers with known probabilistic character. We have also evaluated cross-ApEn and cross-SampEn, which use cardiovascular data sets to measure the similarity of two distinct time series. SampEn agreed with theory much more closely than ApEn over a broad range of conditions. The improved accuracy of SampEn statistics should make them useful in the study of experimental clinical cardiovascular and other biological time series. <s> BIB002 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> There has been considerable interest in quantifying the complexity of physiologic time series, such as heart rate. However, traditional algorithms indicate higher complexity for certain pathologic processes associated with random outputs than for healthy dynamics exhibiting long-range correlations. This paradox may be due to the fact that conventional algorithms fail to account for the multiple time scales inherent in healthy physiologic dynamics. We introduce a method to calculate multiscale entropy (MSE) for complex time series. We find that MSE robustly separates healthy and pathologic groups and consistently yields higher values for simulated long-range correlated noise compared to uncorrelated noise. <s> BIB003 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Traditional approaches to measuring the complexity of biological signals fail to account for the multiple time scales inherent in such time series. These algorithms have yielded contradictory findings when applied to real-world datasets obtained in health and disease states. We describe in detail the basis and implementation of the multiscale entropy (MSE) method. We extend and elaborate previous findings showing its applicability to the fluctuations of the human heartbeat under physiologic and pathologic conditions. The method consistently indicates a loss of complexity with aging, with an erratic cardiac arrhythmia (atrial fibrillation), and with a life-threatening syndrome (congestive heart failure). Further, these different conditions have distinct MSE curve profiles, suggesting diagnostic uses. The results support a general "complexity-loss" theory of aging and disease. We also apply the method to the analysis of coding and noncoding DNA sequences and find that the latter have higher multiscale entropy, consistent with the emerging view that so-called "junk DNA" sequences contain important biological information. <s> BIB004 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Multiscale entropy (MSE) was proposed to characterize complexity as a function of the time-scale factor tau. Despite its broad use, this technique suffers from two limitations: 1) the artificial MSE reduction due to the coarse graining procedure and 2) the introduction of spurious MSE oscillations due to the suboptimal procedure for the elimination of the fast temporal scales. We propose a refined MSE (RMSE), and we apply it to simulations and to 24-h Holter recordings of heart rate variability (HRV) obtained from healthy and aortic stenosis (AS) groups. The study showed that the refinement relevant to the elimination of the fast temporal scales was more helpful at short scales (spanning the range of short-term HRV oscillations), while that relevant to the procedure of coarse graining was more useful at large scales. In healthy subjects, during daytime, RMSE was smaller at short scales (i.e., tau = 1-2) and larger at longer scales (i.e., tau = 4-20) than during nighttime. In AS population, RMSE was smaller during daytime both at short and long time scales (i.e., tau = 1 -11) than during nighttime. RMSE was larger in healthy group than in AS population during both daytime (i.e., tau = 2 -9) and nighttime (i.e., tau = 2). RMSE overcomes two limitations of MSE and confirms the complementary information that can be derived by observing complexity as a function of the temporal scale. <s> BIB005 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> In this work, Refined Multiscale Entropy (RMSE) was applied to characterize risk of cardiac death in ischemic cardiomyopathy patients, analyzing heart rate variability (HRV) by means of RR series during daytime and nighttime. RMSE approach measures an entropy rate in different time scales of a series, giving a multiscale characterization of complexity of that series. RMSE showed statistically significant differences (p<0.05) during daytime and nighttime only in middle time scales (τ=4–15 and τ=3–16, respectively). For these scales, RMSE was higher in low risk (SV) than in high risk (CM) group of cardiac death, indicating a reduction of the entropy-based complexity in CM when it was compared with SV. No statistical differences between risk groups were presented at time scale τ=1 (unfiltered original RR series). It can be concluded that the dynamics in middle time scales should be considered to better describe the HRV of patients with cardiac death. <s> BIB006 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Bearing fault diagnosis has attracted significant attention over the past few decades. It consists of two major parts: vibration signal feature extraction and condition classification for the extracted features. In this paper, multiscale permutation entropy (MPE) was introduced for feature extraction from faulty bearing vibration signals. After extracting feature vectors by MPE, the support vector machine (SVM) was applied to automate the fault diagnosis procedure. Simulation results demonstrated that the proposed method is a very powerful algorithm for bearing fault diagnosis and has much better performance than the methods based on single scale permutation entropy (PE) and multiscale entropy (MSE). <s> BIB007 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Multiscale entropy (MSE) is a prevalent algorithm used to measure the complexity of a time series. Because the coarse-graining procedure reduces the length of a time series, the conventional MSE algorithm applied to a short-term time series may yield an imprecise estimation of entropy or induce undefined entropy. To overcome this obstacle, the modified multiscale entropy (MMSE) was developed. The coarse-graining procedure was replaced with a moving-average procedure and a time delay was incorporated for constructing template vectors in calculating sample entropy. For conducting short-term time series analysis, this study shows that the MMSE algorithm is more reliable than the conventional MSE. <s> BIB008 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Multiscale entropy (MSE) was recently developed to evaluate the complexity of time series over different time scales. Although the MSE algorithm has been successfully applied in a number of different fields, it encounters a problem in that the statistical reliability of the sample entropy (SampEn) of a coarse-grained series is reduced as a time scale factor is increased. Therefore, in this paper, the concept of a composite multiscale entropy (CMSE) is introduced to overcome this difficulty. Simulation results on both white noise and 1/f noise show that the CMSE provides higher entropy reliablity than the MSE approach for large time scale factors. On real data analysis, both the MSE and CMSE are applied to extract features from fault bearing vibration signals. Experimental results demonstrate that the proposed CMSE-based feature extractor provides higher separability than the MSE-based feature extractor. <s> BIB009 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Physiological signals often show complex fluctuation (CF) under the dual influence of temporal and spatial scales, and CF can be used to assess the health of physiologic systems in the human body. This study applied multiscale cross-approximate entropy (MC-ApEn) to quantify the complex fluctuation between R-R intervals series and photoplethysmography amplitude series. All subjects were then divided into the following two groups: healthy upper middle-aged subjects (Group 1, age range: 41-80 years, n = 27) and upper middle-aged subjects with type 2 diabetes (Group 2, age range: 41-80 years, n = 24). There are significant differences of heart rate variability, LHR, between Groups 1 and 2 (1.94 ± 1.21 versus 1.32 ± 1.00, P = 0.031). Results demonstrated differences in sum of large scale MC-ApEn (MC-ApEn(LS)) (5.32 ± 0.50 versus 4.74 ± 0.78, P = 0.003). This parameter has a good agreement with pulse-pulse interval and pulse amplitude ratio (PAR), a simplified assessment for baroreflex activity. In conclusion, this study employed the MC-ApEn method, integrating multiple temporal and spatial scales, to quantify the complex interaction between the two physical signals. The MC-ApEn(LS) parameter could accurately reflect disease process in diabetics and might be another way for assessing the autonomic nerve function. <s> BIB010 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Complex fluctuations within physiological signals can be used to evaluate the health of the human body. This study recruited four groups of subjects: young healthy subjects (Group 1, n = 32), healthy upper middle-aged subjects (Group 2, n = 36), subjects with well-controlled type 2 diabetes (Group 3, n = 31), and subjects with poorly controlled type 2 diabetes (Group 4, n = 24). Data acquisition for each participant lasted 30 minutes. We obtained data related to consecutive time series with R-R interval (RRI) and pulse transit time (PTT). Using multiscale cross-approximate entropy (MCE), we quantified the complexity between the two series and thereby differentiated the influence of age and diabetes on the complexity of physiological signals. This study used MCE in the quantification of complexity between RRI and PTT time series. We observed changes in the influences of age and disease on the coupling effects between the heart and blood vessels in the cardiovascular system, which reduced the complexity between RRI and PTT series. <s> BIB011 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> How the nonstationarity in the atmosphere turbulent vertical velocity series affects its organization degree of multiscale structures is quantified by permutation entropy (PE) and complexity-entropy causality plane (CECP), and marked PE and CECP differences are detected between the nonstationary and stationary series. We find that the value of PE is lower in the nonstationary vertical velocity series than the stationary counterparts. Both types of series locate near the region of the higher complexity value in the CECP as chaotic systems, but the PE is smaller and the complexity degree is larger in the nonstationary series than the stationary with smaller time delays. Due to the close relationship between PE and the multiscale Shannon entropy, we show that the PE and CECP can be also taken as an indicator to quantify the different organization degrees of the multiscale structures existing between the stationary and nonstationary surface vertical velocity records. <s> BIB012 </s> The Multiscale Entropy Algorithm and Its Variants: A Review <s> Original Multiscale Entropy Algorithm <s> Microvascular perfusion is commonly used to study the peripheral cardiovascular system. Microvascular blood flow can be continuously and non-invasively monitored with laser speckle contrast imaging (LSCI) or with laser Doppler flowmetry (LDF). These two optical-based techniques give perfusion values in arbitrary units. Our goal is to better understand the perfusion time series given by each technique. For this purpose, we propose a nonlinear complexity analysis of LSCI and LDF time series recorded simultaneously in nine healthy subjects. This is performed through the computation of their multiscale compression entropy. The results obtained with LSCI time series computed from different regions of interest (ROI) sizes are examined. Our findings show that, for LSCI and LDF time series, compression entropy values are less than one for all of the scales analyzed. This suggests that, for all scales, there are repetitive structures within the data fluctuations. Moreover, at the largest scales studied, LDF signals seem to have structures that are different from those Entropy 2014, 16 5778 of Gaussian white noise. By opposition, this is not observed for LSCI time series computed from small ROI sizes <s> BIB013
The MSE algorithm is composed of two steps BIB003 BIB004 (1) a coarse-graining procedure to derive a set of time series representing the system dynamics on different time scales. The coarse-graining procedure for scale i is obtained by averaging the samples of the time series inside consecutive but non overlapping windows of length i. Thus, for a monovariate discrete signal of length N {x 1 , . . . , x i , . . . , x N } the coarse-grained time series {y (τ) } is computed as  For scale one, the coarse-grained time series {y BIB001 } corresponds to the original signal. The length of the coarse-grained time series {y (τ) } is N/τ. (2) computation of the sample entropy for each coarse-grained time series Sample entropy is a conditional probability measure that quantifies the likelihood that a sequence of m consecutive data points-that matches another sequence of the same length (match within a tolerance of r)-will still match the other sequence when their length is increased of one sample (sequences of length m + 1); m therefore defines the length of the patterns that are compared to each other BIB002 . In this definition, the distance between two vectors is computed as the maximum absolute difference of their corresponding scalar components BIB002 . More precisely, sample entropy is determined as where A m (r) is the probability that two sequences will match for m + 1 points and B m (r) is the probability that two sequences will match for m points (with a tolerance of r), where self-matches are excluded BIB002 . Equation (2) is estimated by the statistics BIB002 SampEn(m, r, By setting we have A/B = A m (r)/B m (r) and the sample entropy can be expressed as BIB002 SampEn(m, r, where A is the total number of forward matches of length m + 1 whereas B is the total number of template matches of length m BIB002 . Regular and periodic signals have a theoretical sample entropy of 0 whereas uncorrelated random signals have a maximum entropy (the value depends on the signal length) BIB002 . The sample entropy for coarse-grained white noise time series can analytically be written as BIB004 − ln where τ and erf( · ) refer to the scale factor and to the error function, respectively. Moreover, Costa et al. have shown that the sample entropy of a coarse-grained 1/f noise is equal to 1.8 for all scales, for the following parameters values: m = 2, r = 0.15× the standard deviation, and N = 30, 000 samples BIB004 . For scale one, white noise time series present a higher value of entropy compared to the one of 1/f time series. Moreover, the value of entropy for the coarse-grained white noise time series monotonically decreases and becomes smaller than the entropy value of coarse-grained 1/f time series for scale factor τ > 4 BIB003 . These results underline that 1/f noise contains complex structures across multiple time scales whereas white noise does not. However, it has been reported that the computation of MSE values as described in the above-mentioned algorithm presents several drawbacks (see, e.g., BIB005 BIB008 ): • The coarse-graining procedure can be seen as a two-steps procedure BIB008 : (I) averaging the data inside a window of length τ in order to reduce the high frequency components; (II) downsampling of the averaged data by a factor τ. The coarse-graining procedure therefore reduces the length of the time series: at a scale factor τ, the coarse-grained time series has a length that is equal to the one of the original time series divided by τ. Therefore, the larger the scale factor, the shorter the coarse-grained time series. It has been reported that to obtain a reasonable entropy value, the time series length should be in the range of 10 m to 20 m BIB002 . For shorter time series, the variance of the entropy estimator grows very fast as the number of data points is reduced. Large variance of estimated entropy leads to a reduction of reliability. Consequently, the statistical reliability of sample entropy for the coarse-grained time series is reduced, as the scale factor τ increases. The sample entropy algorithm therefore leads to an imprecise estimation of entropy-unreliable results with great variance (errors), or even undefined entropy values (when no template vectors are matched to one another)-for short time series or at large time scales. Thus, it has been shown that for synthetic signals for which the theoretical MSE values are known, the estimated MSE values (numerical solutions) may significantly differ from the analytic solutions (see, e.g., BIB009 ). This is particularly annoying for practical applications where it is difficult to obtain long recordings (biomedical field for example). • It has been reported that Equation (1) is similar to the use of a finite-impulse response (FIR) filter BIB005  on the original time series x and to the downsampling of the filtered signal with a factor τ. This FIR filter is a low-pass filter. The authors of BIB005 underline that the features of the frequency response of this low-pass filter are poor, among others because it shows side lobes in the stopband. For the latter reason, the low-pass filter cannot prevent aliasing when the downsampling procedure is applied BIB005 . As a result, the filter does not eliminate fast temporal scale above the filter's cutoff frequency. The downsampling procedure that follows produces aliasing generating spurious oscillations in the frequency between 0 and the filter's cutoff frequency BIB005 . The authors of BIB005 therefore conclude that the evaluation of the complexity of the downsampled time series is biased by these artifacts. • As mentioned previously, in the MSE algorithm two patterns are considered similar if they are closer than a parameter r. The value of r is usually chosen as a percentage of the standard deviation of the signal under study. In the original algorithm proposed by Costa et al., the value of r is constant for all scale factors. However, some authors considered this point as a drawback BIB005 BIB003 . Indeed, as mentioned above, Equation (1) can be seen as a low-pass filtering followed by a downsampling. As a result, when the scale factor increases, the standard deviation of the resulting filtered time series may become lower and lower. Therefore, the patterns may become closer and closer. If the parameter r is constant while the scale factor τ increases, more and more patterns will be considered indistinguishable. This will lead to a decrease of the entropy when the scale factor τ increases. Valencia et al. concluded that MSE measures not only the variations of entropy as a function of the scale factor τ but also the variations in the power of the signal BIB005 BIB006 . On this question, Costa et al. argue that subsequent changes of the variance due to the coarse-graining procedure should be accounted for by the entropy measure BIB004 . From the three points mentioned above and others (see below), several authors proposed modified MSE algorithms in order to overcome the possible drawbacks of the original MSE. These algorithms are described below. MSE is a method that can be used with different types of entropic measures. Thus, some works did not use sample entropy but permutation entropy (see, e.g., BIB007 BIB012 ), cross-approximate entropy (see, e.g., BIB010 BIB011 ), compression entropy (see, e.g., BIB013 ), . . . These works will not be described herein: we will focus only on MSE algorithms with a sample entropy-based approach. However, some of the algorithms described below could be adapted to use not sample entropy but other entropic measures. Moreover, in what follows, the computation of the sample entropy for a time series x with the parameters m and r will be noted as S E (x, m, r).