A generalization of Possibilistic Fuzzy C-Means Method for Statistical Clustering of Data

— The Fuzzy C-means (FCM) algorithm has been widely used in the field of clustering and classification but has encountered difficulties with noisy data and outliers. Other versions of algorithms related to possibilistic theory have given good results, such as Fuzzy C- Means(FCM), possibilistic C-means (PCM), Fuzzy possibilistic C-means (FPCM) and possibilistic fuzzy C-Means algorithm (PFCM).This last algorithm works effectively in some environments but encountered more shortcomings with noisy databases. To solve this problem, we propose in this manuscript, a new algorithm named Improved Possibilistic Fuzzy C-Means (ImPFCM) by combining the PFCM algorithm with a very powerful statistical method. The properties of this new ImPFCM algorithm show that it is not only applicable on clusters of spherical shapes, but also on clusters of different sizes and densities. The results of the comparative study with very recent algorithms indicate the performance and the superiority of the proposed approach to easily group the datasets in a large-dimensional space and to use not only the Euclidean distance but more sophisticated standards norms, capable to deal with much more complicated problems. On the other hand, we have demonstrated that the ImPFCM algorithm is also capable of detecting the cluster center with high accuracy and performing satisfactorily in multiple environments with noisy data and outliers.


I. INTRODUCTION
o classify the data in different groups according to one or more specific criteria, several methods have been adopted to solve this problem, Clustering is one of them, this method uses in most cases the Euclidean distance. There are 2 types of Clustering algorithms, rigid that is based on one object is in a cluster or not with a degree of membership equal to 0 or 1 [1], [2], this type of algorithm has shown its inefficiency in the case of overlapping of two clusters or, in particular, for points belonging to several clusters at the same time, we cite for example the K-Means algorithm. The other type of clustering is Fuzzy Clustering, which is more efficient method, widely used in several fields such as pattern recognition, image processing, also applicable in the security field for face and fingerprint detection, etc..., the principle of this method is simple: several data can belong to different clusters based on multiple membership values from interval [0,1],[3], [4]. Therefore, over the years, many algorithms appeared; the most used being the Fuzzy C-Means (FCM) algorithm [5]. This algorithm was first developed to deal with relational data, derived from Euclidean and other non-Euclidean distances. [6]. FCM algorithm is effective in the case of spherical clusters [7]. However, given the probabilistic type constraint used in FCM, this algorithm encounters the problem of sensitivity to noise and outliers [8], [9]. To correct this weakness, a n ew algorithm called possibilistic C-Means (PCM) which is based on the possibilistic approach was proposed [10]. Also, the PCM algorithm allows a better treatment of noisy data and outliers. Despite its efficiency, the PCM algorithm has a weak point in its sensitivity of initializations and choices of typing parameters and it can sometimes generate coincident clusters [11], another drawback of this algorithm is that it considers the typicities values but neglects memberships values which is also important as a parameter. Other versions of the PCM algorithm have been improved in order to correct some difficulties encountered in the PCM by modifying the objective function [12], [13], [14], [15]. Although PCM only uses typicalities values, the researchers tried conserving the highlights of FCM and those of PCM while mentioning the importance of membership and typicality values. [16], [17]. Hence, a new complementary model called the Fuzzy possibilistic C-Means (FPCM) clustering algorithm have been proposed, which merges the characteristics of the two algorithms FCM and PCM and allows to optimize them [18].
This algorithm has solved some of the problems encountered in the case of FCM and PCM, but like any algorithm, it has its strengths and also its vulnerabilities. The problem with FPCM is that it i mposes a constraint on the typicality values, and also that the possibilistics values are very small when the size of the data set increases, which will lead to the development of a new and more powerful algorithm, called Possibilistic Fuzzy-C Means (PFCM). This new model was first proposed by [16], [19], [20], [21], [22] to simultaneously produce adhesions and typicities, the PFCM is a hybrid combination of the 2 objective functions of the FCM and PCM algorithms, this algorithm has strong points, overcoming the difficulties encountered in FCM, PCM and FPCM, it h as solved the major problem of noise sensitivity encountered in the use of FCM, and that of overlap and coincidence of Clusters, which is the main problem of PCM. The PFCM algorithm was also found to be less sensitive to outliers. Since noise data influence the estimation of centroid, the PFCM algorithm simultaneously creates adhesions and typicities for each cluster with usual prototypes or cluster centers [14], [24], [25], [26].
Although the FCM, PCM, FPCM and PFCM algorithms are based on the Euclidean norm, the problem of using other more efficient norms occurs, for example the covariance norm, which creates ellipsoidal clusters that give better results with models and data structures, then a further problem is encountered, that of compactness, which can lead to the loss of several important data for clusters, such as efficient processing of noisy data or clustering of data with clusters of different sizes [27], [28], [29].
Another problem occurred with these algorithms when using the covariance norm is that FCM, PCM, FPCM and PFCM are unable to provide accurate prototypes even with their own data, in this work, we proposed a new more improved algorithm called ImPFCM (Improved possibilistic uzzy C-Means) to overcome these problems [30].
The ImPFCM algorithm uses functions from the norm; it is more efficient to handle complicated cases encountered as an obstacle for PFCM, especially in the case of noisy data. Hence, ImPFCM algorithm finds accurate cluster centers, its objective function uses norm functions, and it i s flexible, efficient, and more suitable to different clustering concepts and constraints encountered in the above-mentioned algorithms, it is an algorithm capable of using the covariance norm with convenience.
This new algorithm gives a good solution to many of the most problems encountered with other previously mentioned algorithms.
After the introduction, the manuscript is exposed as follow: we start with the description of many clustering algorithms used frequently in clustering theory in sections II, we will present, our proposed improved ImPFCM algorithm, in section III, Finally, we close this manuscript by section IV with some experiments and results of ImPFCM algorithm and conclusion.

II. LITERATURE
In this section, we review the classical existing algorithms used to clustering data we focus attention of our studies to noisy environment with outliers.

A. Fuzzy C-Means algorithm (FCM)
The FCM algorithm is based on the principle of assigning memberships to that are inversely related to the relative distance from to points prototypes these represent the cluster centers in the FCM model. Before the development of the FCM, many algorithms were used such as k-means and C-means, when running these algorithms, the major problem encountered was the treatment of noise and outlying points which can be explained as follows: For two prototypes having the same distance from the center (equidistant), the value of belonging to each Cluster will be identical, whatever the absolute value of the distance between the two centroids (as well as between the other points of the data). Thus, two distant points which belong to the noises but which are the same distance from the Cluster centers, are selected to have equal membership values in the two Clusters, whereas in reality these two points have a very low membership that cancels each other out in relation to one of the Clusters.
In the case of the C -means algorithm, the membership of each data point to all classes is 1, which makes it suffer from the problem of noise sensitivity.
Thus the theory of fuzzy clustering was able to partially solve these problems. The FCM algorithm is derived from the following optimization problem: Minimize: Where: : is number of data vectors, : is number of clusters, : is center of the th i cluster, : is dimension of the data, : is data vector ( column of the data matrix N r X × ), are also the data centers, is the degree of fuzziness, is a norm matrix. This algorithm is characterized by a matrix called the dimensional fuzzy partition matrix , composed of the elements that represent the degree of belonging to the model to each Cluster. is degree of fuzziness, it measures the efficiency of the FCM algorithm on clustering performance [27].
The FCM algorithm has a very special mode of operation when compared to other partitioning algorithms, it works independently of the number of clusters existing in the data set, the FCM algorithm finds a fuzzy partition in a particular set of data. Furthermore, in the FCM algorithm, the sum of each column in the membership matrix U must be equal to 1, this is a main constraint which is the key element for this algorithm that characterizes it to other Clustering algorithms such as the case of C-means and K-means. The FCM algorithm uses data point belongings that are related to the distance of the data point from the cluster centers. If a data point is at the same distance from the clusters, it will have the same membership value for each cluster. However, to deal with noise and outliers, the FCM algorithm is not the right algorithm, since the existence of at least one outlier can completely affect the result of partitioning in the FCM algorithm [8], which are the weak points of the FCM so in this case, the FCM does not differentiate between noise points or outliers which are also taken into account in the membership values which could cause a big problem later on which influences the final result obtained by the FCM. The second problem is that the FCM algorithm only detects spherical clusters. For non-spherical clusters, the FCM becomes inefficient [9].

B. 6 B Possibilistic C-Means algorithm (PCM)
To overcome and correct this weakness FCM algorithm, a new algorithm called PCM based sur possibilistic approach has been proposed [10] and which improves the column sum constraint is equal to 1.

With constraint
In other words, each element of the th column can be any number between 0 and 1, provided that at least one of them is positive. Therefore, the PCM has relaxed this constraint of the FCM and has succeeded in solving the problem of noise sensitivity. However, PCM tends to generate coincident clusters and is very sensitive to initializations [20], [21], [22]. The PCM algorithm is characterized by the degree of typicality with respect to the cluster which is more accurate when compared to the membership values interpreted by the FCM.
As the values of typicality with respect to one group do not depend on any of the prototypes of the other clusters [23], the degrees of typicality have been defined to solve this problem, by constructing prototypes characterizing the subcategories of data, taking into account the particularities of the points with respect to the other categories and also the similarities of the members of the category, therefore, the degree of typicality is an effective means of distinguishing the moderately atypical member of the group from the very atypical member. When compared to the FCM, the PCM algorithm relaxes the line sum constraint of the FCM algorithm, the PCM algorithm is characterized by a main constraint which is expressed as follows: each membership value in can be between 0 and 1 or equal to one of them [24], i.e., . These values are referred to as the data point types in each group. Thus, the objective function of the PCM algorithm can be formulated as follows: With: : the total number of models in a data set, : is the number of Clusters, : The parameter that defines the fuzziness degree of the partition, : the distance which can be Euclidean or not, = : represents the fuzzy partition of the matrix , : the typicity parameter estimated from the data. It is calculated as follows: With: : is the total number of models in a data set is a parameter that defines the degree of Fuziness of the partition; are the characteristics of the data; represent the Cluster's centroid; ] is a fuzzy matrix partition composed of the degrees of membership of the object of each cluster .
For the PCM algorithm, the membership value will be calculated from the following equation: is the distance, is the parameter of typicity. In the case of the PCM algorithm, the value should be interpreted as the typicity of with respect to the cluster (rather than its cluster membership). For PCM, each line can be interpreted as a possibility of distribution on X. The PCM algorithm helps to identify outliers and noisy data points. Although the PCM is efficient in dealing with noise, it has a drawback with its sensitivity of initializations and typing parameter choices and can generate coincident clusters [11]. Also, in the PCM algorithm, clusters do n ot have too much mobility because each data point is classified as a single set at a time, so the PCM produces inaccurate cluster centers when clusters are not of the same size or when the covariance standard is used.
In this way, as it has been observed, several disadvantage exist at the level of the 2 algorithms FCM and PCM, to overcome their weak points, the strong points of the FCM and PCM have been restored, given the importance of the values of typicity and membership, a new model called the Fuzzy possibilistic C-means Clustering FPCM algorithm has been proposed [17] which merges the characteristics of the two algorithms FCM and PCM and allows to optimize them.

C. 7 B Fuzzy Possibilistic C-Means algorithm (FPCM)
The objective function of the FPCM also contains membership and typicality values; however, they are represented as follows in equation (5): Subject to: Where m and η are the coefficients of Fuzziness and Typicity respectively. According to the constraints 6, 7, 8 and the conditions of optimization of the , considering the initial or extreme conditions of and using Lagrange multipliers we obtain the following equations: After running the FPCM algorithm, it was noted that the main problem with the FPCM algorithm is the constraint that corresponds to the sum of all typicality values of all data in the cluster, especially for a large data set [19]. Compared to the FCM, the FPCM algorithm has the same singularity values as the FCM, on the other hand compared to the PCM, the FPCM does not have the sensitivity problem that is a weak point of the PCM, but when the number of data is large, the typicality values (10) will be too low. Thus, typicality values should be scaled up, as in the case of FCM and PCM [25], [26]. Hence, scaling seems to be a p alliative way to solve the problem of small values (which is caused by the line sum constraint on T, since scaled values do not have any additional information on data points. Thus, scaling is a good way to correct a m athematical flaw in the FPCM. This FPCM algorithm is unreliable and also has some defects such as FCM and PCM, to overcome this problem a new algorithm has been introduced called Possibilistic Fuzzy C-Mean. The problem with the FPCM is that it imposes a constraint on the typicities values from the moment when the sum of the typicities values on all data points of a particular cluster is 1, this constraint is relaxed on the typicity values placed normally on the row while keeping the constraint on the membership values placed on the column [27].

D. 8 B Possibilistic Fuzzy C-Means algorithm (PFCM)
The PFCM algorithm was initially proposed by [19], it is an algorithm that has some strong points that overcome the difficulties encountered by FCM, PCM and FPCM algorithms. Principally, the PFCM solves the problem of cluster overlap, it should also be noted that this algorithm is less sensitive to outliers. PFCM is a hybrid combination of the two objective functions of the PCM and FCM algorithms. The objective function of the PFCM is as follows: Under the constraints The coefficients et are weighting values that determine the importance of the typicity values and the membership values N is the total number of the data set, : is the number of Clusters, The coefficients are constants that respectively define the importance of the membership and typicities values in the objective function.
, and are constants defined according to the problem by the user.
is the matrix of typicity values, is considered as an assignment of the typicity of objects in Cluster.
is the matrix of fuzzy partition. is the value of the fuzzy membership function for sample belonging to the Cluster. is the set of data. S: the space dimension.
is the matrix of cluster centers : is the cluster center.
is the distance between and the center of Cluster .
The membership values have the same meaning as those used in the FCM algorithm. Similarly, the values of typicity have the same interpretations as those defined in the PCM model. The constants must respects the following constraint: , and establish the importance of the membership value and the typicity value , Therefore, if we reduce the importance (weight) of the membership value this necessarily forces us to reduce the importance of typicity to the same extent, it is also restrictive. Moreover, to guarantee optimal typicity it will depend on the importance of the value of b. So, by restricting , the flexibility of the model is compromised. If et for every , then equation (12) is reduced to a FCM optimization problem, on the other hand if , then in this case we will be dealing with a PCM optimization problem. We will see further on that if even if we don't set for all , (12) becomes implicitly equivalent to the FCM model. As the FPCM, placed under the regular conditions of C-means optimization problems, we obtain the first order of the necessary conditions for the extrema of J, if we put for any , and contains at least c distinct data points, then minimize if and only if: The FCM, PCM, FPCM and PFCM algorithms are well adapted models to the Euclidean norm, on the other hand the covariance norm creates ellipsoidal clusters which give better results with the models and data structures. Another problem for the Euclidean norm occurs, when the units of the data rows are different, which shows that the Euclidean norm is not well adapted, whereas the covariance norm gives better results [28], [29], [30].
A second problem occurs in the maximum compactness obtained by minimizing the equations (2) and (12) to obtain a simple and linear update equation for the calculation of prototypes et , if we use instead of this requires the resolution of a non-linear equation for the prototypes, however if we use this can lead to the loss of several important cluster data in the case of processing noisy data or clustering data with clusters of different sizes [28]. Other problems encountered with these algorithms when using the covariance norm is that FCM, PCM, FPCM and PFCM are not able to provide accurate prototypes even with clean data, in this paper, we propose a new more general algorithm called ImPFCM to solve some of this shortcomings

III. METHODOLOGY
To make the PFCM more general and efficient, we proposed a new algorithm called Improved Possibilistic Fuzzy C -Means ImPFCM this algorithm is well performing.
The objective function of the IMPFCM is as follows: under the constraints The objective function of the algorithm ImPFCM is almost the same as the one presented in the case of the PFCM, the only existing difference is the replacement of the term in equation (12) by the two terms ) and ).
As we remark, our proposed ImPFCM objective function algorithm is flexible and more scalable efficaciously to the different concepts and constraints of clustering encountered in the previously mentioned algorithms, and which is also able to easily use another norm, for example, covariance norm.
The optimization of equation (16) is performed using Lagrange multipliers for which the following function is optimized. Using Lagrange multipliers, we optimize equation (16) and obtain the following optimized function: The value of the condition is designed to prevent large clusters from pulling the centers of smaller ones, which is discussed in detail in [28], so we calculate the cluster centers as follows: The ImPFCM algorithm converges when with: : index of the iteration : is a predefined threshold.
The membership values and the typicity values are calculated as follows: is calculated from the following derivation: (22) is calculated from the following derivation: IMPFCM is initialized by , , which are calculated from the PFCM algorithm. In the case of the PFCM, based on the equation (15) if , we will not be able to calculate the membership value neither the typicality value , which in our case will be indeterminate values. On the other hand for the IMPFCM algorithm this problem does not occur, so if we impose the condition , in (22), (23) then the values and will always be well determined. In equation (23) if indeed , then , therefore the term in equation (16) will be equal to 0 and equation (16) will become expressed in terms of the Euclidean norm and will have the following form : under contraints Since the identity norm matrix (Euclidean distance) is inappropriate for data sets where the lines have different physical units and interpretations, then as we have already noticed, this equation will subsequently pose a problem in the case cited. This leads us to use the covariance norm matrix in the ImPFCM. For optimal results, the matrix covariance norm was used which gives a dimensionless distance. The ImPFCM algorithm is based on FCM and PFCM in it's initialization and execution, so the steps of ImPFCM are as follows: 1-The FCM is applied. 2-and are calculated to initialize PFCM. 3-After the V's and 's obtained from the PFCM that will be used to initialize the ImPFCM to obtain the most accurate cluster centers. 4-et are calculated from (22) and (23) considering Once these matrices are computed, we notice that they are less sensitive to noise points than those obtained in the PFCM case from (15) because the noise contributions on the cluster centers are reduced using (17).
In other hand, in the case of PFCM, the minimization of the equation gives us a greater compactness, the clustering will be impacted by the noise which influences the quality of the results obtained. Therefore, the functions et help to overcome this problem, they are effective as long as they allow to dampen the noise on the prototypes. The exponential function where is the characteristic width of the Cluster, very appropriate in the case of high noise or distant points. The choice of the functions and influences the convergence of equation (16), that's right why we have to choose them well and avoid functions with low convergence rate which give a too high computation time and wrong results. However, there are other functions that have also shown their ability to attenuate the impact of noise on cluster centers and that can be used instead of the exponential function, for example: In order to avoid high computational time, it should be noted that the type of functions used affects the convergence of equations (16)- (17) and functions with low convergence rates should be avoided for the reason cited above.

A. Comparative study of FCM, PFCM, ImPFCM and algorithms
In this section, we focus on the experimental analysis and testing ImPFCM performance algorithm, we need to compare it with FCM and PFCM algorithms using simple dataset containing noise and taken from [19] and presented as following in Table I: The six datasets are presented as below in Fig. 1. With various ratios of noise points to the actual data points: Fig. 1 The Six Datasets used for testing proposed clustering algorithms To compare the performance of the three algorithms FCM, PFCM and ImPFCM, in terms of exactness to detect more precisely the clusters centers, we perform a clustering on these 6 datasets, we start with data 1. Table II  and Table III, we can deduce that FCM algorithm is not efficient to find the cluster center exactly, because it detects the approximate location of the prototypes near the potential clusters, while the PFCM algorithm reduces the noise effects and finds the approximate clusters centers, the ImPFCM method localizes the centers precisely compared to FCM and PFCM algorithms, Fig. 2. According to the results provided in Table IV and Table V, we can observe that the ImPFCM algorithm successfully detects the cluster centers accurately, while the PFCM algorithm performs well then FCM algorithm, but the ImPFCM algorithm performs satisfactorily despite the noisy environment. Fig. 3. Results of clustering DATA 2 using FCM, PFCM and ImPFCM, c=4, m=2 In data 2 clustering results, from Table IV, Table V and Fig.3, it's clear that ImPFCM algorithm outperformed in comparison with FCM and PFCM algorithms and the ImPFCM error is 88% smaller than FCM error. In addition, the ImPFCM error is 13% smaller than PFCM error. *For Data 3: Cluster centers and FCM, PFCM, ImPFCM algorithms performances for data 3 are presented Hence, for data 3, with a number of cluster centers equal to 6, and from the obtained tests results in Table.VI, Table VII, Fig.4, we can conclude that the ImPFCM algorithm is more performing to identifying exactly the cluster centers versus the FCM and PFCM algorithm, and, the ImPFCM error is 87% less than that of FCM and 19% less than PFCM error. *For Data5: FCM, PFCM, ImPFCM algorithms Cluster centers and performances are presented in following Table IIX  and Table IX. Fig.5.

PFCM ImPFCM
Once again with six clusters, we observed that DATA 5 is clustered Fig. V. When using the ImPFCM algorithm, we notice that the cluster centres found are identical to the actual centres, Table IIX, Table IX, thus the ImPFCM accurately detected all six cluster centeres. Thereby, the ImPFCM again demonstrates superiority and performance over both the FCM and PFCM algorithms. As we remark, the ImPFCM error is 92% less than that of FCM, and 25% less than PFCM error.  Fig. 4 Result of clustering DATA 3 using FCM, PFCM and ImPFCM, c=6, m=2 for data 3 Table IIX. Matrix of initial and FCM, PFCM, ImPFCM cluster centers for data 5 In order to investigate the ImPFCM algorithm performance in various environments, we scope the execution of the ImPFCM algorithm to other types of databases, and accordingly, we perform clustering on 6 clean and noiseless datasets and 13 real datasets using the FCM, PFCM and ImPFCM algorithms and then we compare them in both number of iterations and execution time, results are presented in following (Fig.6, Table  X): Fig. 6 Six synthetic data sets with no noise and outliers In a further extension of the study to other types of databases, the FCM, PFCM and ImPFCM algorithms were applied to 25 synthetic and real datasets and compared in terms of execution time and number of iterations, as detailed below.
-Data sets 1 to 6 are the six synthetic noisy data sets studied above and shown in Fig. 1, Fig.3, Fig.4, and Fig.5.
-Data sets 7 to 19 are real data taken from the UCI Machine Learning Repository.
-Data sets 20 to 25 are the synthetic noise-free shown in Fig. 6 that contain different numbers of clusters to study the algorithms performance when there are large numbers of clusters in the data.
To examine the reliability of our ImPFCM algorithm, we started by measuring the efficiency of ImPFCM with thirteen real-world datasets with a known number of clusters, and then including previously known datasets such as: Climate Model Simulation Crashes, Connectionist Bench, Glass ID, Energy Efficiency, Fertility, Glass ID, Haberman Survival, Cleveland Heart Disease, Ionosphere, IRIS, Pima Indian Diabetes, Seeds, Wine, and Wisconsin Prognostic Breast Cancer, which are all taken from the UCI Machine Learning Repository. In the case of real datasets, distributed between 7 and 19, and due to outliers, the FCM algorithm remains unable to detect cluster centres characterised by their large size, but in this case the PFCM finds dense clusters, but the ImPFCM algorithm is more efficient on this point compared to the PFCM algorithm, so it can detect large clusters with higher densities and presents remarkable accuracy results compared to the PFCM and FCM algorithms. In another hand, there is no noise or outliers in the 20-25 datasets and their clusters have the same sizes. Therefore, the ImPFCM algorithm easily finds the actual cluster centers of these sets and also iterates only twice because the datasets are clean, and due to the interactions between the clusters Fig.VI, it corrects the insignificant displacements of the cluster centers calculated by the PFCM algorithm. Based on the results in Table X, it deduced that the PFCM and FCM algorithms require relatively less execution time and iterations than the ImPFCM algorithm, this is due to the projection into another space and the non-linear cluster center update equation, two major reasons that influence the execution time of the ImPFCM algorithm, On the other hand, according to what it noticed, sometimes, even if there is no noise or outliers in the data, FCM and PFCM algorithms remain limited in this case and cannot find the exact cluster centers due to the interactions between clusters, ImPFCM algorithm limits or cancels these interactions due to function introduced in the objective function and finds the cluster centers precisely. Hence, we can deduce that ImPFCM algorithm detects the cluster centers more precisely. We can deduce that, when using ImPFCM algorithm, although we lost some time but we gain in terms of accuracy.
B. The impact of Cluster sizes and density to ImPFCM algorithm The quality of the clustering is indicated by the density, so a good clustering is equivalent to a high density which means that the cluster centers calculated by the algorithm are located in dense regions of the data. Therefore, we have a h igher density rho, when there are many data points around each cluster center, which is expressed using the following equation.
In order to evaluate the impact of different cluster sizes on the cluster centers, we notice that the cluster size influences the location of the cluster centers. Thus, when the cluster sizes are varied, the cluster centers are misplaced, and this even in noise-free datasets, because the larger clusters attract the cluster centers to their side. For this reason, if we experiment with the FCM, PFCM, and ImPFCM algorithms on a noisefree dataset composed of two clusters of different sizes as shown in Fig.7, we notice that, the FCM algorithm places the two cluster centers in the larger cluster. On the other hand, the PFCM algorithm identifies the center of the largest cluster accurately and the center of the smallest one as well, the ImPFCM algorithm, is initialized by the cluster centers computed by the PFCM algorithm, this algorithm also shows its efficiency in this point, cancelling the mutual interactions of the clusters and finding the real location of the cluster c.

C. Computational cost of the ImPFCM algorithm
Although the ImPFCM algorithm shows its performance and effectiveness to detect precisely cluster centers, this algorithm in its execution, is greedy in terms of consumption of execution time, and convergence, and finding the precise center of the clusters requiring a higher computational cost compared to the FCM algorithm, and this is due to the nonlinear nature of the equations contained in the objective function and their update, the high execution time of the ImPFCM algorithm compared to FCM, is caused by the complex and excessive computations for convergence, which is not the case for FCM and also because the functions are linear and do not require additional calculations. On the other hand, compared to algorithm PFCM, we notice that for testing FCM , PFCM and ImPFCM algorithms to some data set for example chosen Data 1 to data 6, it is noted that when the program start and the number of iteration was higher in threshold 7, Fig.8. the ImPFCM algorithm has relatively a small error of convergence compared to PFCM algorithm that explain a high precision to find exactly a real clusters centers, finding accurate cluster centers in presence of noise and outliers and finding accurate cluster centers when sizes of the clusters are considerably different, and this is due to using statistical method, so we can deduce that the ImPFCM algorithm improves PFCM and FCM algorithms and works satisfactorily. In summary, the main contribution of the present paper is to improve performance of the well-known FCM and PFCM algorithms by proposing the ImPFCM algorithm. As mentioned in this manuscript and in other researches done previously, the existence of noise in the data, the precise determination of the center of the clusters, the outliers, the density, the different sizes of the clusters, the interactions between the clusters, have always constituted great obstacles for the clustering method, Therefore, several algorithms have been developed to solve these problems, for example as mentioned in this paper among others the algorithms FCM, PCM, FPCM, PFCM these last ones have not been able to determine the real centers of the clusters, and have not given satisfactory results, So in order to overcome these difficulties, a new algorithm named ImPFCM has been proposed which represent an improved version of PFCM, subsequently, it was also shown in this work that the proposed ImPFCM algorithm is more efficient and accurate, when the data is noisy and can accurately calculate the real centers of the clusters when the size of the clusters is significantly different and when there are interactions between these clusters, and, In particular, the ImPFCM algorithm finds the dense regions of the data more precisely when compared to the FCM and PFCM algorithms, which is indicated by the density of the clustering results, and it converges faster than the ImPFCM algorithm. Furthermore, the results of the comparative study algorithms indicates the performance and efficiency of ImPFCM algorithm to easily cluster the data sets in a space with a high dimension and to use not only Euclidean distance but more sophisticated norms able to deal with much more complicated problems. On the other hand, the ImPFCM algorithm performs satisfactorily even when the covariance norm matrix is used, in which case the FCM, PCM, PFCM algorithms fails to find accurate prototypes even in clean data, while the ImPFCM is insensitive to the cluster size and the type of covariance norm matrix, and work effectively in different environments with noisy data and outliers.
1 2 B Acknowledgment The author gratefully acknowledge the all members of scientific team which participated in this research.