a support vector machine ( svm ) is a distinguishing classifier conventionally defined by a separating hyperplane . in other words ,given labeled training data ( supervised learning ) , the algorithm outputs an optimal hyperplane which classifies the unseen examples[1 - 2 ] .svm is not limited statistics or machine learning but can be applied in wide number of applications .svm proved to be the best classifier in different applications from handwritten digit recognition to text categorization .svm does nt have any effect on classification due to the curse of dimensionality .it works well with high dimensional data .literature survey witnesses that the support vector machines are the best classifiers for 2-class classification problems .the real time applications are not limited to binary classification but multiclass classification[3 ] .for example , to classify an astronomical object as a star , a galaxy or a quasar requires a multi - class classifier but not binary classifier .there are two major approaches in solving the n - class problem one is a single large optimization problem [ 4 - 5 ] , and the alternative is to decompose n - class problem into multiple 2-class problems . but solving a single large optimization problem will be expensive in terms of computational time and not suitable for practical applications .there are several algorithms based on the second approach like ovo(one - versus - one),ova(one - versus - all ) , dag(directed acyclic graph)[6 - 7 ] . in this paperwe propose a new algorithm which decomposes the single n - class problem into multiple 2-class problems . andit requires less number of binary classifiers when compared with the above mentioned algorithms and gives a better accuracy too .the rest of the paper is organized as follows .the related work is discussed in section 2.the proposed approach is presented in section 3 .section 4 contains the experimental results and comparison of the proposed approach with the existing algorithms .finally , section 5 contains conclusion and future work .the learning task in binary svm can be represented as the following   subject to where and are the parameters of the model for total number of instances .+ using lagrange multiplier method the following equation is to be solved ,   + the dual version of the above problem is + subject to  where are known as the lagrange multipliers . + by solving this dual problem , svm will be found .once the svm model is built , the class label of a testing object z can be predicted as follows .+ if z will be predicted as + class else -ve class .the simple approach is to decompose the problem of classifying n classes into n binary problems , where each problem differentiates a given class from the other n-1 classes [ 8 ] . in this approach ,we require k = n binary classifiers , where the nth classifier is trained with positive examples belonging to class n and negative examples belonging to the other n-1 classes .when an unknown example is to be predicted , the classifier achieving the maximum output is considered as the best choice , and the corresponding class label is assigned to that test object .though this approach is simple [ 8 ] , it provides performance that is comparable to other more complicated approaches when the binary classifier is tuned well .   in this approach , each class is compared to every other class [ 9 - 10 ] .a binary classifier is built to differentiate between each pair of classes , while discarding the rest of the classes .this requires building n(n-1)/2 binary classifiers .when testing a new object , a voting is performed among the classifiers and the class with the maximum number of votes will be considered as the best choice .results [ 6,11]show that this approach is in general better than the one - versus - all approach .the n - class problem is decomposed into multiple 2-class problems in a binary tree structured manner .k - means clustering is used as a preprocessing step to get the rough estimation of similarity between the class labels .this will let us divide the class labels into two disjoint sets and build the svm for the root node .thereafter every node is divided at the mid point for creating disjoint sets .the order of the class labels is computed based on the sse .the least sse will be first in the list and the highest sse will appear last in the list .this way ( n-1 ) binary svms will be built , and hence needs only ( n-1)/2 svms evaluation to classify the unclassified record .this is better than the worst case ( n-1 ) of ova and n(n-1)/2 of ovo . at the same time ,our experimental results show that the accuracy is comparable with ovo .but when cbts is compared with ova it shows a better accuracy with reduced training time and testing time .the algorithm for training and testing is illustrated below .input the training objects .add all the training objects to the root node .let the class labels are from 1 .... n + * preprocessing step : * divide the training objects i.e.the root node into two clusters / nodes ( ) and ( ) using k - means clustering ( centroid based ) .+  1 .  the objects will be adjusted to ( ) as positive class or ( ) as negative class based on the majority of their class labels from the two clusters .+ 2 .  for ( ) and ( ) calculate the sse of all objects based on the same class labels and sort them in the ascending order .the sse is given by + 3 .for both ( ) and ( ) repeat + 1 .if the number of class labels of the node are two , construct the binary classifier and return . if the number of class labels are more than two + 2 .divide the each node exactly at the mid point , construct the binary svm and repeat this till we reach only two class labels .   1 .the test object should be evaluated on the root node the binary tree of svms .repeat : * if the value is positive traverse to the left node ( ) else to the right node ( ) . + * until we reach the leaf node .classify the test object into the class label of leaf node .+  * let s run through an example .* consider figure 1 , suppose if we have 8 classes , we first run k - means clustering with k=2 to divide the data objects according to their distribution .then , through the cluster distribution and based on the majority we get to know which class labels fall on one side ( positive ) and which ones on the other side ( negative ) . in the example shown , set \{1,3,7 } belong to the positive class and set of rest of class labels , \{4,6,8,2,5 } belong to the negative class. the order of class labels within the node will be in the ascending order of sse .we then build a svm model by constructing a binary svm between data objects belonging to as belonging to positive class and those belonging to as negative class .now , for the left child of the root node , sample space is all the data objects that belong to class labels in i.e. \{1,3,7}. we divide exactly at the mid point ( remember , clustering is done at the first step only ! ) and hence make new and sets for this node .so , new is \{1,3 } and is \{7}. we now consider all data objects having class labels in as having positive class and all in as negative class and build a svm model .this svm model acts as left child of the root node .  this way we build the whole binary tree of svm functions recursively for both left and right nodestill we reach a leaf node .when an unseen object has to be classified , the search starts from the root node and then it moves on to the left or right based on the evaluation function value recursively till the leaf node and assigns the corresponding class label to the test object .      in figure 2 , the hyperplanes are shown for different class labels based on the above approach .the order of the construction of the hyperplanes will be based on the binary tree structure .we implemented the algorithm on twelve classification data sets and compared the training time , testing time and accuracy with ovo and ova .glass , iris , letter , mfeat - fac , mfeat - fou , mfeat - kar and mfeat - mor , pendigits , satimage , segment , shuttle , vowel are uci data sets and taken from https://archive.ics.uci.edu/ml/datasets.html . the detailed properties of the data sets are given in table i. for letter , shuttle and satimage dataset , a separate training and testing file was used .but , for the rest of all the data sets 2/3 part of data is considered for training and 1/3 is considered for testing .all the datasets were scaled to [ 0,1 ] and have been randomized ( similar class labels data will not appear together ) . in our implementation , we make use of radial basis function(rbf ) kernel and the range of value of  to , c  to .rbf kernel works best for any kind of problem [ 16 ] . in all the three approaches ovo , ova ,cbts the best combination of and c is chosen from the above mentioned range so as to provide the higher accuracy and lower testing and training time [ 17 ] .all the computations are done on a computer with 1.60ghz intel i5 - 4200u dual core processor and ram of 6 gb and using the software libsvm [ 17 ] .  in tableii , the training time and testing time of cbts is compared with ovo and ova .cbts performs very well when compared with ova .though the training time and testing time of cbts is little more than ovo , and cost values are justified to have a good model .  in table iii ,the results show that the accuracy of all twelve datasets by cbts are comparable with ovo with less cost c. a higher value of c means to choose more samples as support vectors . but limiting the support vectors i.e.limiting the value of c will use the minimum memory possible and the prediction will be faster . even when the is compared , cbts is choosing the intermediate values as for ovo it is too small . for a good model ca nt be too small or too large . if is too small the model is restricted and can not capture the complexity or `` shape '' of the data . if is too largethere is a possibility that model will become over fit the data [ 18 ] . coming to the comparison with ova, cbts gives a better accuracy .  in table iv , the number of binary classifiers required for one classification is shown for ovo , ova and cbts .if we observe cbts requires the minimum number of binary classifiers when compared with ovo and ova .in addition to above data sets we analyzed the sloan digital sky survey ( sdss ) data set also .sdss is a major multi - filter imaging and spectroscopic redshift survey using a dedicated 2.5-m wide - angle optical telescope at apache point observatory in new mexico , united states , astronomical telescope survey .it has 6 class labels and only 5 features ( u , g , r , i , z ) are considered ( it has many ) .the data can be downloaded from sql interface on http://skyserver.sdss.org/dr7/en/tools/search/sql.asp  the results are shown in table v for sdss data set .cbts outperforms both ova and ovo in the aspects of accuracy and training time with the best gamma and cost values .the size of the instances are varied from 30,000 to 75,000 .ova could nt build the binary classifiers when the data size is 75,000 because of scalability issue whereas ovo and cbts could build the required models with out any problem .hence cbts is scalable . and it gives a better accuracy with reduced training time ..the complete details of the data sets used in our implementation . [ cols="^,^,^,^",options="header " , ]we propose a new algorithm cbts ( centroid based binary tree structured svm ) , a binary tree structure in which the root node contains all the class labels and is splitted based on the k - means clustering at the first step .and then the left and right nodes are recursively splitted into half till we have only two class labels . in this n-1 svmsare required to construct but the required svms in ovo , ova are n(n-1)/2 , n respectively .the testing time for all ovo , ova , dag is linear on n i.e. o(n ) but for our algorithm it is o(log n ) as it needs only ( n-1)/2 classifiers to predict the class label for the test object .experimental results show that the accuracy of cbts is comparable with ovo approach and it out performs with ova both in accuracy and training , testing time .cbts also capable of handling large data sets , hence scalable . by using a more optimized approach of implementing cbts , testing time may further be reduced . instead of k - meansclustering alternative partition techniques can also be explored at the root level .furthermore , a parallel / distributed version of our algorithm can be taken up as a future work . in our algorithmthere are independent svm models that can be constructed simultaneously which results in reducing both training and testing time of cbts even further .we are thankful for the resources provided by the department of computer science and information systems , bits , pilani , k.k .birla goa campus to carry out the experimental analysis .c.cortes and v. vapnik ._ support vector network .1em plus 0.5em minus 0.4emseptember 1995 , volume 20 , issue 3 , pp 273 - 297 v. vapnik . _the nature of statistical learning theory_. ny : springer - verlag .k.crammer,y.singer , _ on the learnability and design of output codes for multiclass problems_. 1em plus 0.5em minus 0.4emproceedings of the thirteenth annual conference on computational learning theory , morgan kaufmann publishers inc . , san francisco , ca , usa,2000,pp.35 - 46 v.n.vapnik _ statistical learning theory_. 1em plus 0.5em minus 0.4emjohn wiley and sons , new york,1998 .y.guermeur , combining discriminant models with new - multi - class svms , 1em plus 0.5em minus 0.4empatt.anal.appl.5(2002 ) 168 - 179 chih - wei hsu and chih - jen lin ._ a comparison of methods for multi class support vector machines_. 1em plus 0.5em minus 0.4emieee transactions on neural networks , vol. 13 , no .2 , march 2002 abe , shigeo _ analysis of multi support vector machines_. 1em plus 0.5em minus 0.4emproc .international conference on computational intelligence for modelling control and automation ( cimca2003 ) , : 385 - 396 ryan rifkin , aldebano klautau , _ in defense of one vs - all classification _1em plus 0.5em minus 0.4emjournal of machine learning research 5 ( 2004 ) 101 - 141 ._ another approach to polychotomous classification . technical report _ ,1em plus 0.5em minus 0.4em stanford university , 1996 .trevor hastie and robert tibshirani ._ classification by pairwise coupling ._ 1em plus 0.5em minus 0.4em advances in neural information processing systems , volume 10 .the mit press,1998 erin allwein , robert shapire , and yoram singer . _ reducing multiclass to binary : a unifying approach for margin classifiers_. 1em plus 0.5em minus 0.4em journal of machine learning research , pages 113141 , 2000  l.bottou , c. cortes , j. denker , h. drucker , i. guyon , l. jackel , y. lecun , u. muller , e. sackinger , p. simard , and v. vapnik ._ comparison of classifier methods : a case study in handwriting digit recognition_. 1em plus 0.5em minus 0.4emin international conference on pattern recognition , pages 77 - 87 .ieee computer society press , 1994 .s. knerr , l. personnaz , and g. dreyfus , _ single - layer learning revisited : a stepwise procedure for building and training a neural network_. 1em plus 0.5em minus 0.4em neurocomputing : algorithms , architectures and applications .springer - verlag , 1990 ._ another approach to polychotomous classification . technical report ,department of statistics , stanford university,1996_. 1em plus 0.5em minus 0.4emavailable at http://www-stat.stanford.edu/reports/friedman/poly.ps.z u. krebel ._ pairwise classification and support vector machines_. 1em plus 0.5em minus 0.4emadvances in kernel methods support vector learning , pages 255 - 268 , cambrdige , ma , 1999 . mit press .chih - chung chang and chih - jen lin ._libsvm : a library for support vector machines ._ 1em plus 0.5em minus 0.4em acm transactions on intelligent systems and technology , 2:27:127:27 , 2011 .software available at http://www.csie.ntu.edu.tw/ cjlin / libsvm

Abstract: support vector machines ( svms ) were primarily designed for 2-class classification . but they have been extended for n - class classification also based on the requirement of multiclasses in the practical applications . although n - class classification using svm has considerable research attention , getting minimum number of classifiers at the time of training and testing is still a continuing research . we propose a new algorithm cbts - svm ( centroid based binary tree structured svm ) which addresses this issue . in this we build a binary tree of svm models based on the similarity of the class labels by finding their distance from the corresponding centroids at the root level . the experimental results demonstrates the comparable accuracy for cbts with ovo with reasonable gamma and cost values . on the other hand when cbts is compared with ova , it gives the better accuracy with reduced training time and testing time . furthermore cbts is also scalable as it is able to handle the large data sets .   k - means clustering / centroid based clustering ; svm ; multi - classification ; binary tree ;


the probability density function ( pdf ) is a key concept in statistics with many practical applications , see for example , .constructing the most adequate pdf from the observed data is still an important and interesting research problem , especially for large datasets .pdfs are often calculated using nonparametric data - driven methods .one of the most popular nonparametric method is the kernel density estimation ( kde ) .however , a very serious drawback of using kde is the large number of calculations required to compute them as well as to find the optimal bandwidth ( smoothing ) parameter ( time complexity ) .  in this paperwe investigate the possibility of utilizing field - programmable gate arrays ( fpga ) to accelerate finding of such the optimal bandwidth . towards the needs of the paperwe have selected one popular and often used algorithm called in literature the plugin .this work can be considered as a continuation and extension of the paper , where the authors utilize gpus for speeding up optimal bandwidth selection .one of the algorithm analyzed in that paper was the above mentioned plugin one .  generally , there are two methodologies for speeding up complex numerical algorithms : software - based and hardware - based . in this paperwe concentrate only on hardware - based methods .the commonly known approaches are as follows : ( a ) computing on general purpose multicore cpu microprocessors , ( b ) computing on distributed environments ( e.g. clusers , grids , etc . ) , ( c ) computing on gpus , ( d ) computing on dsp units and ( e ) computing on fpga chips .  in the paper we are concerned with fpga approach . in author considers a problem how to use fpga for fast computing of pdfs using direct vhdl programming approach .however , the problem we are concerning is of different nature as we concentrate our attention for computing the optimal bandwidth for pdf ( see chapter [ sec : kde - band ] ) .  to develop the final fpga design we use the high level synthesis ( hls ) approach , , where no direct hardware description language ( hdl ) coding is needed ( typically in vhdl or verilog languages ) .the remainder of the paper is organized as follows . in section [ sec : kde - band ] we turn our attention to give the reader some preliminary information on kde and bandwidth selection . in section [ sec : plugin ] we give detailed mathematical formulas for calculating optimal bandwidth using the plugin method . in section [ sec : fpga - impl ] we cover all the necessary details on our fpga - based implementation .we also present practical experiments we carried out and discuss the results . in section [ sec : concluding ] we conclude the paper .let us consider a continuous _ univariate _ random variable and let assume its probability density function exists but is unknown .its estimate , usually denoted by , will be determined on the basis of a random sample of size , that is ( our experimental data ) . in such a case , a -dimensional kernel density estimator of a real density for random sample given by the following formula  is a positive real number called _ smoothing parameter _ or _ bandwidth _ and is the _ kernel function _ a symmetric function that integrates to one . in practical applications has often the gaussian normal form , that is if we have the bandwidth , we can determine the estimator of the unknown density function using formula ( [ eq : kde - def ] ) .the bandwidth is the parameter which exhibits a strong influence on the resulting kde .formula ( [ eq : kde - def ] ) can be easily extended to the _ multivariate _ case . in the most general variantthe scalar bandwidth is replaced by the _ unconstrained _ bandwidth matrix ( which is symmetric and positive definite ) . also ( [ eq : gaussian ] )is generalized to the multivariate case .two commonly used kernel types are _ product _ and _ radial _ ( also known as _ spherically symmetric _ ) ones .as an example of how kde works consider a toy dataset of 8 data points : three different kdes based on these data are depicted in figure [ fig : kernel - demo ] .it is easy to notice how the bandwidth influences the shape of the kde curve .lines in bold show the estimated pdfs , while normal lines show the shapes of individual kernel functions ( gaussians ) .dots represent the data points .    choosing the best value of is not a trivial task and this problem was and still is extensively studied in literature , .currently available selectors can be roughly divided into 3 classes , .the first class uses very simple and easy to calculate mathematical formulas .they were developed to cover a wide range of situations , but do not guarantee being enough close to the optimal bandwidth .they are often called rules - of - thumb methods .the second class contains methods based on least squares and cross - validation ideas with more precise mathematical arguments , but they require much more computational effort .however , in reward for it , we get bandwidths more accurate for a wider range of density functions.  the third class contains methods based on plugging in estimates of some unknown quantities that appear in formulas for the asymptotically optimal bandwidth .one selected method from the third class is investigated in the paper .the method is briefly presented in chapter [ sec : plugin ] and from now on it will abbreviated as the plugin .in algorithm [ alg : plugin ] we give recipe for calculation of the optimal bandwidth using the plugin method ( the symbols used are exactly such as in the book ) .all the necessary details on the method , as well as details on deriving of particular mathematical formulas can be found in many source materials , see for example books .it is important to stress that the plugin algorithm is a strictly _ sequential _ computational process ( see figure [ flowchart : of : plugin ] ; parallel processing is possible only internally in steps iv and vi ) as every step depends on the results obtained in the previous steps .first we calculate the variance and the standard deviation estimators of the input data , see step i in algorithm [ alg : plugin ] .then we calculate some more complex formulas from step ii to step vi .finally , we can substitute them into equation given in step vii to get the searched optimal bandwidth value .* step i : * _ calculate the estimates of variance ( ) and standard deviation ( _ ) :   * step ii : * _ calculate the estimate of functional _ :   * step iii : * _ calculate the bandwidth of the kernel estimator of function ( 4th derivative of function , that is ) _ : * step iv : * _ calculate the estimate of functional _ : ,  \\k^{6}(x )   & = \frac{1}{\sqrt{2\pi}}\left ( x^6 - 15x^4 + 45x^2 -15 \right ) e^{-\frac{1}{2}x^2}.  \end{split}\ ] ]  * step v : * _ calculate the bandwidth of the kernel estimator of function _ : * step vi : * _ calculate the estimate of functional _ : ,  \\k^{4}(x )   & =   \frac{1}{\sqrt{2\pi}}\left ( x^4 - 6x^2 + 3 \right ) e^{-\frac{1}{2}x^2}.  \end{split}\ ] ]  * step vii : * _ calculate the final value of the bandwidth _ :     our implementation of the algorithm [ alg : plugin ] is carried out in fixed - point arithmetic ( see section [ sec : impl - prelim ] ) . unfortunately , using the raw data while conducting the required calculations , threatens a potential problems with overflow , especially while calculating the value of , see step ii in algorithm [ alg : plugin ] .note that the estimate of standard deviation in is raised to the power of . for large values of results in extremely small values of .the above problems can be successfully overcome if the input datasets are _ standardized _ using the _ z - score _ formula , that is where and are mean and standard deviation of the original vector respectively ._ z - score _ guarantees that in and , consequently , entity has simply a constant value . applying the data standardizationrequires an extra operation on the value in step vii in algorithm [ alg : plugin ] , that is where is the bandwidth calculated for the standardized dataset and is the standard deviation of the original vector .the correctness of the above equation can be easily proofed algebraically .  to reduce the calculation burden we can also slightly change equations and in algorithm [ alg : plugin ] .it is easy to notice a symmetry , that is   so , the double summations can be changed and , consequently , the final formula for has now the following form \label{eq : plugin - psi6estimate-2}\ ] ] ( note for different summation ranges , the , , 2 before sums and an extra factor added , that is ) . obviously , the same concerns and  .\label{eq : plugin - psi4estimate-2}\ ] ]  computation complexity of steps iv and vi ( double summations ) , where the symmetry property is used , still belongs to complexity class where , and represents computation time for the differences , represents division time , and represents time for computing and polynomials .high level synthesis ( hls ) is an automated design process that interprets an algorithmic description of a problem ( given in high level languages c / c++ ) and translates this problem into a so called register - transfer level ( rtl ) hdl code .then in turn this hdl code can be easily synthesized to the gate level by the use of a logic synthesis tool , like for example _ xilinx ise design suite _ , _ xilinx vivado design suite _ , _ altera quartus ii_. in this paper we discuss results obtained using a tool called _ xilinx vivado high level synthesis _ , a feature of _ vivado design suite_. this tool supports c / c++ inputs , and generates vhdl / verilog / systemc outputs .other solutions are offered by _ scala _ programming language and a specialised high level synthesis language called _cx _ .it should also be mentioned that a similar tool called a++ is also available for altera fpga devices .   before implementing the plugin algorithm [ alg : plugin ] it is important to take some assumptions affecting both performance and resource consumption .  _the first assumption _ is about a proper arithmetic used .the floating - point one gives very good range and precision .unfortunately , from fpga s point of view , this representation is very resource demanding .in contrast , the fixed - point arithmetic is much less resource demanding but its range and precision are more limited .hence , the exact fixed point representation was determined based on a careful analysis of the particular intermediate values taken during calculations . if the input dataset does nt contain extremely large outliers ( which suggests that such dataset should be first carefully analyzed before any statistical analysis taken ) and if the _ z - score _ standardization is used , fixed point representation is sufficient for all calculations ( that is : integer part length , fractional part length , word length and the first bit represents the sign ) .note also that as a result of the _ z - score _ standardization , the vales of , , are constant and this significantly simplifies the calculations .the fractional part does give the required precision .however , the integer part must also be sufficiently large , as factors are present in the plugin algorithm . _ the second assumption _ is about choosing the most adequate methods for calculating individual steps in algorithm [ alg : plugin ] .now it needs to be stressed that programming for fpga devices differs considerably from programming for cpus / gpus devices .fpga devices are built from a large number of simple logical blocks like : look up tables ( lut ) , flip - flops ( ff ) , block rams ( bram ) , specialized dsp units ( dsp ) .these blocks can be connected each other and can implement only relatively low - level logical functions ( the so called gates level ) . as a consequence , even very basic operations , like for examples the adder for adding two numbers must be implemented from scratch . in description of the plugin algorithm [ alg : plugin ]one can easily indicate such operators like ( a ) addition , ( b ) subtraction , ( c ) multiplication , ( d ) division , ( e ) reciprocal , ( f ) exponent , ( g ) logarithm .] , ( h ) power , ( i ) square roots , ( j ) higher order roots .our implementation utilizes the following methods : _ cordic _ , for calculating exponents and logarithms ,divisions were replaced by multiplications and reciprocals , difference operators were replaced by addition of negative operands . additionally , one extra implementation of the exponent function was used for calculations of and in algorithm [ alg : plugin ] .this implementation is based on the remez algorithm , , and is open to pipelining . as a consequence ,a significant speedup can be achieved during calculations of steps iv and vi in algorithm [ alg : plugin ] .it is also worth to note that the authors implementation of the division operator ( base on multiplications and reciprocals ; the reciprocal is based on the newton method ) is significantly faster than the default division operator available in vivado hls .moreover , the another advantage of using our own operators , is that no ipcore ( xilinx s library of many specialized functions available for fpga projects ) is needed . as a consequence ,the generated vhdl codes are more portable for fpga chips from different than xilinx vendors .  _ the third assumption _ during implementing of the plugin algorithm was to enable the nominal clock frequency of an fpga chip used ( see chapter [ sec : impl - results ] for details ) . during experimentsit was turned out that the usage of the original division operator resulted in problems with reaching the required frequency .the authors original implementation of the division operator ( base on multiplications and reciprocals ) solved this problem .  _ the forth assumption _ was that all the input datasets must be stored in the bram memory , which are available in almost all current fpga chips .they have enough capacity to store truly large data , like even 500,000 elements or more .   in figure[ fig : plugin - unitis ] we show the scheme of the plugin implementation where all the main components are presented .they correspond literally to the seven steps shown in algorithm [ alg : plugin ] .    figure [ fig : one - block - architecture ] presents general architecture of the functional unit for computing ( step vi in algorithm [ alg : plugin ] ) .it is worth to note that the proper architecture of this unit must be reached during careful coding in vivado hls , using techniques like listed in section [ sec : impl - prelim ] .unit at the block - level view .the extra frame called _ ul - part _ shows the part of the step vi in algorithm [ alg : plugin ] where _ loop unrolling _ can be used , width=491 ]  we developed three different versions of the plugin algorithm .  _the first implementation _ , called _ literal _ , is just a literal rewriting of algorithm [ alg : plugin ] ( with the improvements ( [ eq : plugin - psi6estimate-2 ] ) and ( [ eq : plugin - psi4estimate-2 ] ) ) .no additional actions were taken toward optimization of both execution time and resource requirements .this version can operate with any unscaled input data ( assuming that all the inputs as well as all the internal results fulfil the fixed - point ranges that have been set ) .this version automatically ( vivado decides ) utilizes pipelining .however , the pipelining does nt make the implementation enough fast and additionally , large number of dsp blocks is used .ffs and luts usage is also quite big ( see table [ tab : resource - usage ] ) .  _ the second implementation _ , called _ minimal _ , is written so that it is optimized for resource utilization , mainly the dsp units . to reduce the number of the dsp units some dedicated functions for addition and multiplicationare required .using vivado hls compiler s pragmas ( _ # pragma hls inline off _ ) pipelining can be disabled ( on default , during translation of the high level codes into hdl ones pipelining is enabled whenever it is possible ) .as can be observed in table [ tab : resource - usage ] , a significant reduction of the dsp units was achieved .it confirms the fact that vivado hls is very sensitive for the structure of the high level codes being translated into hdl ones .so that , to achieve good performance and resource usage many modifications of the high level codes are required .  _ the third implementation _ , called _ fast _ , is written so that it is optimized for time execution .addition and multiplication functions were implemented in two ways . in the first way ( similar as in _ minimal _ implementation )the pipelining is disabled , while in the second way it is enabled .the pipelined versions of the functions are used in steps iv and vi in algorithm [ alg : plugin ] as these two steps are crucial for the final performance .additionally , in these two steps a dedicated implementation of the exponent function was used ( based on remez algorithm which is more likely to pipelining ) .also , a technique known as _ loop unrolling _ was used in a manual manner ( see sample codes in figure [ fig : sample - loops ] ) .although vivado hls uses automatic loop unrolling , this feature does nt work correctly in our algorithm ( as it can operate with datasets of any size and the exact number of loops is not known in advance ) .the _ forth and fifth implementations _ used during experiments ( called _ cpu _ and _ gpu _ respectively ) are the ones implemented and investigated in .cpu implementation utilizes the sse ( streaming simd extensions ) of the current multicore cpus .   during all practical experiments the target _ xilinx virtex-7 xc7vx690tffg1761 - 2 _device was used .its nominal working frequency is 200mhz ( or 5 ns for a single clock tact ) .cpu implementation was run on _ intel processor i7 4790k 4.0 ghz_. _ geforce 480gtx _ graphics card was used for gpu implementation ._ vivado hls ver .2015.2 _ was used for developing all the fpga implementations .the summary of the _ resource consumption _ is given in table [ tab : resource - usage ] .additionally , _ power consumption _ is included .it is a real power ( in watts ) taken by the fpga chip after physical implementation of the plugin algorithm using vivado design suite .the power consumption of the fpga implementations is significantly smaller comparing with the power consumption of the cpu and gpu implementations .the power consumption for the cpu and gpu used in our experiments are an average ( catalogue - like ) values ..resources usage for three different fpga implementations of the plugin algorithms as well as cpu and gpu implementations .additionally power consumption is included . [cols="^,^,^,^,^,^ " , ]   the summary of the _ accuracy _ for three different implementations of the plugin algorithm is given in table [ tab : accuracy ] . is the reference bandwidth calculated in double floating point arithmetic ( in c++ program , 1517 significant decimal digits ) .it is worth to note that the relative errors for _ literal _ , _ minimal _ and _ fast _ implementations are very small ( not more than ) . in practical applicationssuch small values can be in fact neglected .cccc * n & & & * + 128 & 0.304902711650357 & 0.304902701728222 & 3.25e-06 + 256 & 0.227651247521862 & 0.227651285449348 & 1.67e-05 + 384 & 0.202433198224753 & 0.202433187549741 & 5.27e-06 + 512 & 0.242707096505910 & 0.242707026022425 & 2.9e-05 + 640 & 0.190442902734503 & 0.190443702342891 & 0.00042 + 768 & 0.175199386896566 & 0.175199406819444 & 1.14e-05 + 896 & 0.172251554206014 & 0.172251524317464 & 1.74e-05 + 1024 & 0.174044180661440 & 0.174044236921001 & 3.23e-05+ * *  * n & & & * + 128 & 0.304902980336919 & 0.304902701728222 & 9.14e-05 + 256 & 0.227651586290449 & 0.227651285449348 & 0.000132 + 384 & 0.202433346537873 & 0.202433187549741 & 7.85e-05 + 512 & 0.242707266006619 & 0.242707026022425 & 9.89e-05 + 640 & 0.190443017752841 & 0.190443702342891 & 0.000359 + 768 & 0.175199396442622 & 0.175199406819444 & 5.92e-06 + 896 & 0.172251742798835 & 0.172251524317464 & 0.000127 + 1024 & 0.174044403014705 & 0.174044236921001 & 9.54e-05 + *n & & & * + 128 & 0.304901758907363 & 0.304902701728222 & 0.000309 + 256 & 0.227651913650334 & 0.227651285449348 & 0.000276 + 384 & 0.202433891594410 & 0.202433187549741 & 0.000348 + 512 & 0.242707268567756 & 0.242707026022425 & 9.99e-05 + 640 & 0.190443484811112 & 0.190443702342891 & 0.000114 + 768 & 0.175199736841023 & 0.175199406819444 & 0.000188 + 896 & 0.172251721611246 & 0.172251524317464 & 0.000115 + 1024 & 0.174044031649828 & 0.174044236921001 & 0.000118 + * * * *  the summary of the _ scalability _ of different plugin algorithm implementations is presented in figure [ fig : fpga - implementation - speedup ] .scalability of the fpga implementations is nearly linear , which is a very welcome behavior .the corresponding results for _ cpu _ and _ gpu _ implementations can be found in .the figure is in fact a graphical summary of data given in table [ tab : execution - times ] .axis is used),width=453 ]  simplified source codes of the three fpga implementations are presented in figure [ fig : sample - loops ] .complete source codes ( c++ and resulted vivado hls translations into vhdl ) are available on request .the first version is just the literal implementation of the step vi in algorithm [ alg : plugin ] in c language .unfortunately , as can be observed in table [ tab : execution - times ] and in figure [ fig : fpga - implementation - speedup ] such implementation is very slow . in the second version multiplications and additions are realized using dedicated functions ( _ fadd _ , _ fmul _ ) .also a dedicated function for reciprocal operator was implemented . in the third versionmuch more modification was implemented .first , loop unrolling was used , second , vivado hls pragmas were used and third , multiplications and additions were realized using dedicated functions with pipelining enabled ( _ pfadd _ , _ pfmul _ ) .  .... //literal implementation psi4_f1 : for ( i=0 ; i < n ; i++ ) {   psi4_f2 : for ( j = i+1 ; j < n ; j++ ) {     s = s + k4 ( ( ( x[i ] - x[j ] ) / g2 ) ) ;   } }  // minimal implementation rg2 = reciprocal ( g2 ) ; psi4_f1 : for ( i=0 ; i <n ; i++ ) {   psi4_f2 : for ( j = i+1 ; j <n ; j++ ) {     s = fadd ( s , k4 ( fmul ( fadd ( x[i ] , -x[j ] ) , rg2 ) ) ) ;   } }  // fast implementation rg2 = reciprocal ( g2 ) ; psi4_f1 : for ( i=0 ; i < n ; i++ ) {   psi4_f2 : for ( j = i+1 ; j < n ; j+=2 ) {          # pragma hls expression_balance     # pragma hls pipeline          if ( j = = i+1 ) tmp = 0.0 ;     if ( j < n ) { tmp1 = 0.0 ; tmp2 = 0.0 ; }     psi4_f1_b0 : {       tmp1a = pfadd ( x[i ] , -x[j ] ) ;       tmpva = pfmul ( tmp1a , rg2 ) ;       tmp1 = k4 ( tmpva ) ;     }     psi4_f1_b1 : {       if ( ( j+1 ) <n ) {         tmp1b = pfadd ( x[i ] , -x[j+1 ] ) ;         tmpvb = pfmul ( tmp1b , rg2 ) ;         tmp2 = k4 ( tmpvb ) ;       }     }     if ( j < n ) { tmp = pfadd ( tmp , tmp1 ) ; tmp = pfadd ( tmp , tmp2 ) ; }     if ( j+2>=n ) s = pfadd ( s , tmp ) ;   } } ....hls tools are competitive with manual design techniques using hdls .implementation time of complex numerical algorithms can be essentially reduced ( comparing to direct coding in hdl languages ) .unfortunately , to obtain efficient fpga implementations , many changes to source codes are required , comparing to equivalent implementations for cpus and/or gpus .this is because fpga devices use specific primitives like dsp , bram , ff and lut blocks and programmers should control their utilization manually .however , this control is performed on the level of c / c++ codes , not the hdl ones .it is also worth to stress that using the hls approach allows to obtain implementations which are often faster than cpu and/or gpu counterparts .another crucial motivation for replacing gpu or cpu solutions by their fpga equivalents is power consumption .fpga can settle for single watts , while cpu or gpu counterparts typically take tens / hundreds of watts or even more .we would like to thank for useful discussions with colleagues at the institute of control and computation engineering ( issi ) of the university of zielona gra , poland .we would like also to thank to anonymous referees for useful comments on the preliminary version of this paper .the numerical results were done using the hardware and software available at the fpga / gpu -lab located at the institute of control and computation engineering of the university of zielona gra , poland .andrzejewski , w. , gramacki , a. , gramacki , j. : graphics processing units in acceleration of bandwidth selection for kernel density estimation .869885 ( 2013 )  bachrach , j. vo , h. , richards , b. lee , y. , waterman , a. , avizienis , r. , wawrzynek , j. , asanovi , k. : chisel : constructing hardware in a scala embedded language , in : design automation conference , ieee , pp .1212 1221 , san francisco ( 2012 )           lei , y. , dou , y. , dong , y. , zhou , j. , xia , f. : fpga implementation of an exact dot product and its application in variable - precision floating - point arithmetic .j. supercomput .64 , issue 2 , pp . 580605 ( 2013 )    pedro ferlin e. , silvrio lopes h. , erig lima , c.r . ,perretto , m. : prada : a high - performance reconfigurable parallel architecture based on the dataflow model , int . j. of high performance systems architecture , vol . 3 , no . 1 ,41 - 55 ( 2011 )        taherkhani , s. ; ever , e. ; gemikonakli , o. , : implementation of non - pipelined and pipelined data encryption standard ( des ) using xilinx virtex-6 fpga technology , computer and information technology ( cit ) , in : 10th international conference on computer and information technology , pp .1257 1262 , ieee press , bradford ( 2010 )

Abstract: fpga technology can offer significantly higher performance at much lower power consumption than is available from cpus and gpus in many computational problems . unfortunately , programming for fpga ( using hardware description languages , hdl ) is a difficult and not - trivial task and is not intuitive for c / c++/java programmers . to bring the gap between programming effectiveness and difficulty the high level synthesis ( hls ) approach is promoting by main fpga vendors . nowadays , time - intensive calculations are mainly performed on gpu / cpu architectures , but can also be successfully performed using hls approach . in the paper we implement a bandwidth selection algorithm for kernel density estimation ( kde ) using hls and show techniques which were used to optimize the final fpga implementation . we are also going to show that fpga speedups , comparing to highly optimized cpu and gpu implementations , are quite substantial . moreover , power consumption for fpga devices is usually much less than typical power consumption of the present cpus and gpus .   * keywords : * fpga , high level synthesis , kernel density estimation , bandwidth selection , plug - in selector


massive mimo is regarded as a promising technology in future wireless networks due to its high spectrum and energy efficiency .the large spatial degree of freedom ( dof ) of massive mimo systems can contribute to ( i ) spatial multiplexing gains for intra - cell users per bs ( mu - mimo ) as well as ( ii ) inter - cell interference mitigation between the bss via linear precoders at the bss . in ,zero - forcing ( zf ) and regularized zero - forcing ( rzf ) have been proposed for spatial multiplexing of data streams to intra - cell users .more complicated linear precoding schemes based on duality or semidefinite relaxing ( sdr ) have also been proposed to achieve a better performance . on the other hand ,the inter - cell interference mitigation between bss is more complicated .one commonly adopted approach to mitigate the inter - cell interference is the coordinated mimo strategy , which performs joint precoding among the bss using the global real - time csis shared among the bss . alternatively , cooperative mimo techniques can also be exploit to mitigate inter - cell interference by sharing both real - time csi and payload data among the concerned bss .however , these conventional spatial multiplexing and interference mitigation techniques can not be applied directly to massive mimo cellular networks due to the following reasons .first , the mu - mimo precoding requires real - time local csit at the bs .however , the amount of pilot symbols for channel estimation is limited by the coherence time of the channel and it is practically infeasible to obtain good csi quality when each bs is equipped with a massive mimo array .second , the existing inter - cell interference mitigation methods such as cooperative and coordinated mimo require real - time global csit , which is difficult to achieve in practice due to the backhaul latency . hence , the performance of these schemes is very sensitive to csit errors due to outdatedness .  in this paper , we address the above issues by proposing a hierarchical interference mitigation scheme for massive mimo cellular networks . in the proposed scheme , the mimo precoder at each bs is partitioned into an _ inner precoder _ and an _outer precoder _ as illustrated in fig .[ fig : hybridpc ] .the inner precoder is used to support mu - mimo ( control intra - cell interference and capture the spatial multiplexing gain ) at each bs and it is adaptive to real - time local csit .the outer precoder can leverage on the remaining spatial dof to mitigate the inter - cell interference by restricting the transmitted signal at each bs into a subspace and is adaptive to long - term channel statistics . ] .such hierarchical precoding structure simultaneously resolves both the aforementioned practical challenges .for instance , the issue of insufficient pilot symbols for real - time local csi estimation is resolved because the bs only needs to estimate the csi within the subspace determined by the outer precoder , which is of a much smaller dimension than the number of antennas .furthermore , the outer precoder is adaptive to the long - term channel statistics , which is insensitive to backhaul latency . as a result , the proposed _hierarchical precoding _framework exploits the spatial dof to simultaneously achieve spatial multiplexing per bs and inter - cell interference mitigation without expensive backhaul signaling requirement .we consider joint optimization of the outer precoders , the user selection , and the power allocation to maximize a general concave utility function of the average data rates of users .the following first - order challenges need to be addressed .  * * lack of closed - form optimization objective * : the average data rate of each user involves stochastic expectation over csi realizations and it does not have closed form characterization . * * complex coupling between user selection and outer precoding * : the outer precoder will affect the admissible user set .on the other hand , the optimization of outer precoder also depends on user selection because the outer precoder only needs to suppress the interference to the selected users in other bss . * * combinatorial optimization problem * : the user selection problem with hierarchical precoding in the massive mimo cellular networks is combinatorial with exponential complexity w.r.t .the total number of users .  to address the above challenges ,we first apply the random matrix theory to obtain an approximated problem with closed - form objective . then using the hidden convexity of the problem, we propose an iterative algorithm to find the optimal solution for the approximated problem .we also obtain a low complexity algorithm with provable convergence .finally , we illustrate with simulation that the proposed design achieves significant performance gain compared with various state - of - the - art baselines under various signaling backhaul latency ._ notations _ : the superscripts and denote transpose and hermitian respectively . for a set , denotes the cardinality of .the operator represents a diagonal matrix whose diagonal elements are the elements of vector .the notation denote the set of all semi - unitary matrices .let denote the indication function such that if the event is true and otherwise . represents the subspace spanned by the columns of a matrix and represents a set of orthogonal basis of . is the spectral radius of .consider the downlink of a massive mimo cellular network with bss and single - antenna users as illustrated in fig .[ fig : sysmodel ] for and .each bs has antennas with much larger than the number of the associated users .denote as the channel vector between bs and user .the channel fading process is modeled as , where has i.i.d .complex entries of zero mean and variance ; and is the spatial correlation matrix between bs and user .the random process is quasi - static within a time slot but i.i.d .time slots , user and bs indices .the spatial correlation process is assumed to be a slow ergodic process ( i.e. , remains constant for a large number of time slots ) according to a general distribution . as such, the csi is divided into instantaneous csi and global statistical information ( spatial correlation matrices ) . due to local scattering ,the spatial correlation matrices of different users in cell is usually different .however , if the coverage area of a bs is partitioned into small sub - areas , it is reasonable to assume that any two users collocated in the same sub - area have almost the same spatial correlation matrices .this motivates us to consider the following locally - clustered spatial channel model .[ asm : chmodel]the spatial correlation matrices associated with bs belongs to a finite set with the size .furthermore , due to the local spatial scattering , we have .  assumption [ asm : chmodel ]is realistic because in practice , there are only limited number of significant eigenvalues in a mimo channel ( especially for large ). the massive mimo cellular network can be represented by a topology graph as define below .[ network topology graph ] for given spatial correlation matrices , define the _ topology graph _ of the massive mimo cellular network as a bipartite graph , where denotes the set of all bs nodes , denotes the set of all user nodes , and is the set of all edges between the bss and users . for each bsnode , let denote the set of associated users and denote the set of neighbor users . for each usernode , let denote the index of its serving bs and denote the set of neighbor bss .define =\textrm{tr}\left(\mathbf{\theta}_{k , n}\right) ] is the data symbol vector at bs ; and {l\in\mathcal{s}_{n}}\in\mathbb{r}_{+}^{\left|\mathcal{s}_{n}\right|} ] is the precoding matrix at bs ; and is the awgn noise .   conventional interference mitigation techniques for small scale mimo cellular networks such as mu - mimo precoding , coordinated mimo , or cooperative mimo , can not be applied directly to massive mimo cellular networks due to two practical challenges , namely , the insufficient pilot symbols for csi estimation and the backhaul latency . to resolve these practical challenges , we propose a novel hierarchical interference mitigation control , which can fully utilize the large number of antennas to simultaneously mitigate the inter - cell interference as well as realize the spatial multiplexing gain per bs .specifically , the interference mitigation strategy is partitioned into _ long - term _ and _ short - term _ controls .the short - term control is responsible to capture the spatial multiplexing gain among the intra - cell users at each bs based on the local csit only . on the other hand ,the long - term control is responsible to mitigate the inter - cell interference based on the global statistical information .they are elaborated as follows .we first use a simple example to illustrate the idea of hierarchical precoding .an example massive mimo cellular network with one ring scattering model to illustrate the role of spatial correlation matrices .the direct link channel subspace is illustrated in subfigure ( b ) .it consists of two orthogonal subspaces and .,width=264 ]  [ exm : txspace - motivation]consider the massive mimo cellular network in fig .[ fig : exmphych]-(a ) .each bs has antennas .consider the one ring model in for transmit antenna correlation , where a user is surrounded by a ring of scatterers such that the support of its angle - of - departure ( aod ) distribution are restricted to a certain region as illustrated in fig .[ fig : exmphych]-(a ) .assume that user 1 and user 2 share the same scattering ring and are restricted in the same 4-dimensional subspace ( i.e. , ) as illustrated in fig .[ fig : exmphych]-(b ) .moreover , due to the local scattering configuration as illustrated in fig .[ fig : exmphych]-(a ) , the cross link channel vector between bs 1 and user 3 is restricted in a 2-dimensional subspace ( i.e. , ) .we consider a hierarchical precoder structure for bs 1 : , where with is the outer precoder adaptive to the spatial correlation matrices only , and is the inner precoder adaptive to the local csi ( the effective channel ^{\dagger}\in\mathbb{c}^{2\times2} ] .define as the set of outer precoders for all bss . by properly choosing ( equation ( [ eq : icizf ] ) ), one can eliminate the inter - cell interference as illustrated in example [ exm : txspace - motivation ] .physically , the rank of means the number of data streams for spatial multiplexing at bs . due to limited spatial scattering ,a bs with say antennas does not mean it can support spatial multiplexing of 100 data streams . in practice , there are just a few ( say ) significant eigenchannels despite having 100 antennas , and having spatially multiplexed data streams already capture most of the spatial multiplexing advantage .the remaining spatial dofs can be used for inter - cell interference mitigation .an illustration of hierarchical precoder structure.,width=264 ]  for a given outer precoder , we consider regularized zero - forcing ( rzf ) inner precoder with a parameter .the rzf precoder is easy to implement and is asymptotically optimal for  .moreover , we can apply the technique of deterministic equivalent ( de ) for rzf precoding in to facilitate the algorithm design . for convenience ,define the composite channel from bs to any subset of users as {l\in\mathcal{u}^{s}}^{\dagger}\in\mathbb{c}^{\left|\mathcal{u}^{s}\right|\times m} ] as will be shown in lemma [ lem : asyws ] .as such , the user selection set and power allocation is assumed to be adaptive to the global statistical information only . specifically , the user selection and the outer precoder are chosen to satisfy the _zero inter - cell interference constraint _ : and the power allocation has to satisfy a per - bs power constraint that will be elaborated later .note that * * the inter - cell interference from bs to a user can be expressed as . since is positive semidefinite , setting * * is equivalent to setting .hence , the constraint in ( [ eq : icizf ] ) can be used to eliminate the inter - cell interference to all users in the system . in ( [ eq : icizf ] ) , we consider zf criteria for inter - cell interference mitigation due to its simplicity and asymptotic optimality at high snr .similar zf criteria has also been used in to design pre - beamforming matrix based on spatial correlation matrices in single cell systems .[ implementation considerations][rem : lcsi]in the proposed hierarchical interference mitigation , the long term controls such as outer precoding , statistical user selection and power allocation are implemented at a central node based on the global statistical information about the channel ( ) , while the short term control ( inner precoding ) is implemented locally at each bs based on the local real - time instantaneous csi knowledge ( ) .the proposed hierarchical precoding solution has several unique benefits regarding implementation .( a ) robust to csi signaling latency in backhaul , ( b ) resolve the issues of insufficient pilot and feedback overhead for real - time csit estimation in massive mimo systems .for instance , the central node requires the global statistical information ( spatial correlation matrices ) to compute the long term controls .the spatial correlation matrices can be estimated via downlink training using some standard covariance matrix estimation technique at the users and then fed back to the bss . since changes at a much slower time scale w.r.t .the time slot rate , such a design requires substantially less signaling overhead compared to the coordinated mimo and is more robust w.r.t . to the backhaul latency . on the other hand ,bs only needs to know the local real - time instantaneous csi for the inner precoder .this can be obtained via downlink channel estimation and channel feedback using conventional csi signaling mechanisms in modern wireless systems such as lte .since is substantially smaller than , the issue of the huge downlink pilot and csi feedback signaling overhead in massive mimo is also alleviated by the hierarchical precoding design .we consider joint optimization of the outer precoders , the user selection , and the power allocation ^{t} ] with the inner precoder given by ( [ eq : rzfvn ] ) .the transmit power of bs is note that there may not always be enough spatial dofs to eliminate the inter - cell interference to all the users .hence , for a fixed composite control variable , it is possible that only part of the users can be scheduled for transmission . for fairness considerations ,we consider randomized control policy which realizes time - sharing between several composite control variables as defined below .  [ randomized control policy][def : randomized - control - policya]a randomized control policy consists of a set of composite control variables with and a probability vector ^{t} ] . at any time slot, the composite control variable is used with probability , i.e. , the outer precoders , the user selection set and the power allocation are respectively given by , and with probability .moreover , define the set of feasible control policies under per - bs power constraint as where \leq p_{c}\right\ } ] is the conditional average rate vector .we make the following assumptions on ( is a simplified notation for ) .[ asm : ufun]the utility function can be expressed as , where is the weight for user , is assumed to be a twice differentiable , concave and increasing function for all .moreover , is l - lipschitz , i.e. , for some constant .the above utility function captures a lot of interesting cases :  * * alpha - fair : * alpha - fair can be used to compromise between the fairness to users and the utilization of resources .the utility function is is equal to zero . in this paper , we set so that assumption [ asm : ufun ] can be satisfied . since is very small, it has negligible effect on the performance .the utility function in ( [ eq : alffair ] ) is also scaled by to ensure that it is bounded as . ] * * where is a small number . * * proportional fair ( pfs ) : * this is a special case of alpha - fair when .  for a given topology graph and per - bs power constraint ,the problem of interference mitigation via hierarchical precoding can be formulated as depends on since the set of neighbor users of bs depends on . ] :   note that the conditional average rate in the utility function and the conditional average power  ] with . then we have the following lemma .[ convexity of [lem : convexity - of - r] , where denotes the convex hull operation and ^{t}:\gamma\in\xi^{\textrm{f}\circ}\left(p_{c}\right)\right\ } ] is optimal for problem ( [ eq : equplapre-1 ] ) if and only if   finally , from lemma [ lem : hiddencvx ] and lemma [ lem : first - order - optimalityhd ] , we can obtain the necessary and sufficient global optimality condition for problem as follows .[ global optimality condition of ][thm : global - optimality - conditionpe]a control policy with is a global optimal solution of if and only if satisfies : where ^{t} ] .  the detailed proof can be found in appendix [ sub : proofs - for - the - gop ] .   just as we can obtain the optimal solution of a convex problem by solving its kkt conditions , we can also obtain the global optimal solution of by solving the global optimality condition in theorem [ thm : global - optimality - conditionpe ] .specifically , for any given spatial correlation matrices , we propose algorithm e to achieve the global optimality condition of by iteratively updates the optimization variables and the weight vector _ _ in theorem [ thm : global - optimality - conditionpe ].  _ algorithm e _( top level algorithm for solving ) :  * * initialization * * : set and let {k=1, ... ,k} ] .let and {j\in\mathcal{j}^{(i)}} ] and {i\in\mathcal{s}_{n}}\in\mathbb{r}^{\left|\mathcal{s}_{n}\right|} ] , {i\in\mathcal{s}_{n}}\in\mathbb{r}^{\left|\mathcal{s}_{n}\right|} ] given by   following similar analysis as in the proof of ( * ? ? ?* theorem 3 ) , it can be shown that and .then it follows that and . from this and lemma [ lem : asysinr ], lemma [ lem : asyws ] follows immediately .let be the optimal solution of problem .it can be proved by contradiction that the control policies and must satisfy : , and .define two sets let denote a control policy that satisfies , , and .let denote a control policy that satisfies , , and .it can be shown that as , we have for or .for composite control variable satisfying the conditions in lemma [ lem : asyws ] , it can be shown that and are uniformly integrable w.r.t . . together with lemma [ lem : asyws ], it follows that -r_{k}^{\circ}\left(\gamma|\mathbf{\theta}\right)\right| & \leq & o\left(\nu\right),\label{eq : rgap0a}\\ \underset{m\rightarrow\infty}{\textrm{lim } } \left|\textrm{e}\left[\left.p_{n}\left(\gamma\right)\right|\mathbf{\theta}\right]-p_{n}^{\circ}\left(\gamma|\mathbf{\theta}\right)\right| & \leq & o\left(\nu\right),\label{eq : pgap0a}\end{aligned}\ ] ] by definition , we have then it follows from ( [ eq : pgap0a ] ) and ( [ eq : pgap0 ] ) that -p_{c}\leq o\left(\nu\right),\:\textrm{as}\ : m\rightarrow\infty.\label{eq : pgapf1}\ ] ] similarly , it can be shown that we expand as follows +\nonumber \\ \left[u\left(\overline{\mathbf{r}}\left(\hat{\omega}^{*}|\mathbf{\theta}\right)\right)-u_{e}\left(\hat{\omega}^{*}\right)\right]+\left[u_{e}\left(\hat{\omega}^{\star}\right)-u\left(\overline{\mathbf{r}}\left(\hat{\omega}^{\star}|\mathbf{\theta}\right)\right)\right].\label{eq : gapi}\end{aligned}\ ] ] from ( [ eq : rgap0a ] ) , we have for .then it follows from ( [ eq : romga ] ) and that from ( [ eq : omgconv],[eq : pgapf2 ] ) and the definition of and , we have then it follows from ( [ eq : omgconv],[eq : gapi],[eq : gapi0],[eq : iequ ] ) that this completes the proof for theorem [ thm : asyeqp ] .[ [ proofs - for - the - results - in - section - subglobal - optimality - conditionsubproofs - for - the - gop ] ] proofs for the results in section [ sub : global - optimality - condition][sub : proofs - for - the - gop ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~   clearly , . hence , we only need to prove that any pareto boundary point of must lie in .first , it is easy to see that can always be expressed as a convex combination of points in , i.e. , , where  ] and .hence , must lie in .the first part of lemma [ lem : hiddencvx ] follows directly from the definition of problem ( [ eq : equplapre-1 ] ) and .the second part of lemma [ lem : hiddencvx ] can be proved by contradiction .suppose satisfies but is not the global optimal solution of .then there exists a control policy such that . then compared to , achieves a larger objective value for problem ( [ eq : equplapre-1 ] ) , which contradicts with the assumption that is the optimal solution of problem ( [ eq : equplapre-1 ] ) .suppose with satisfies the optimality condition in theorem [ thm : global - optimality - conditionpe ] .it follows from ( [ eq : optcondpe ] ) that and . using the above fact and noting that , where ,\forall j\:\textrm{and}\:\sum_{j=1}^{\left|\xi\right|}q_{j}^{\star}=1 ] is given by clearly , we have where the equality holds if and only if . from ( [ eq : u1]-[eq : tau ] ) , we have . by lemma [ lem :property - of - algorithme ] , we have . hence by lemma [ lem : algcp1 ] , we have then it follows from ( [ eq : uetao1 ] ) and ( [ eq : uelim ] ) that combining ( [ eq : taoi ] ) and the fact that , we have .this completes the proof .[ [ proof - of - theorem - thmconve_greedyaesubproof - of - theoremgreedyae ] ] proof of theorem [ thm : conve_greedyae][sub : proof - of - theoremgreedyae ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  using similar analysis as in the proof of lemma [ lem : property - of - algorithme ] , it can be shown that under the modified algorithm e. since the objective value is upper bounded , we have for some . following similar analysis as thatfor ( [ eq : taoi ] ) , it can be shown that any accumulation point of the iterates generated by the modified algorithm e satisfies moreover , it follows from that .let denote the optimal solution of .since is the gradient of ( by definition ) and is a concave function , we have where the last inequality follows from and ( [ eq : mepro ] ) .f. rusek , d. persson , b. k. lau , e. larsson , t. marzetta , o. edfors , and f. tufvesson , `` scaling up mimo : opportunities and challenges with very large arrays , '' _ ieee signal processing magazine _ , vol .30 , no . 14060 , jan . 2013 .c. peel , b. hochwald , and a. swindlehurst , `` a vector - perturbation technique for near - capacity multiantenna multiuser communication - part i : channel inversion and regularization , '' _ ieee trans ._ , vol .53 , no . 1 ,195 202 , jan . 2005 .g. foschini , k. karakayali , and r. valenzuela , `` coordinating multiple antenna cellular networks to achieve enormous spectral efficiency , '' _ iee proceedings on communications _ , vol .153 , no . 4 , pp .548 555 , aug .h. zhang and h. dai , `` cochannel interference mitigation and cooperative processing in downlink multicell multiuser mimo networks , '' _ eurasip journal on wireless communications and networking _ , vol .2004 , no . 2 , pp . 222235 , 2004 .s. wagner , r. couillet , m. debbah , and d. t. m. slock , `` large system analysis of linear precoding in correlated miso broadcast channels under limited feedback , '' _ ieee trans .info . theory _ ,58 , no . 7 , pp . 45094537 , jul. 2012 .h. lei , l. zhang , x. zhang , and d. yang , `` a novel multi - cell ofdma system structure using fractional frequency reuse , '' _ in proc .ieee int . symp .personal , indoor mobile radio commun ._ , pp . 15 , sep . 2007 .o. somekh , o. simeone , y. bar - ness , a. haimovich , and s. shamai , `` cooperative multicell zero - forcing beamforming in cellular downlink channels , '' _ ieee trans .inf . theory _ ,55 , no . 7 , pp .32063219 , 2009 .

Abstract: we propose a hierarchical interference mitigation scheme for massive mimo cellular networks . the mimo precoder at each base station ( bs ) is partitioned into an _ inner precoder _ and an _ outer precoder_. the inner precoder controls the intra - cell interference and is adaptive to local channel state information ( csi ) at each bs ( csit ) . the outer precoder controls the inter - cell interference and is adaptive to channel statistics . such hierarchical precoding structure reduces the number of pilot symbols required for csi estimation in massive mimo downlink and is robust to the backhaul latency . we study joint optimization of the outer precoders , the user selection , and the power allocation to maximize a general concave utility which has no closed - form expression . we first apply random matrix theory to obtain an approximated problem with closed - form objective . we show that the solution of the approximated problem is asymptotically optimal with respect to the original problem as the number of antennas per bs grows large . then using the hidden convexity of the problem , we propose an iterative algorithm to find the optimal solution for the approximated problem . we also obtain a low complexity algorithm with provable convergence . simulations show that the proposed design has significant gain over various state - of - the - art baselines .   massive mimo , hierarchical interference mitigation , statistical user selection


the mechanisms of adaptation and learning in natural and artificial systems are the subject of paramount interest in neuroscience .most of the experimental evidence point to the synaptic modification as the physiological basis of the long - term memory .hebb was the first who suggested that the synaptic coupling between two neurons is enhanced if both neurons are simultaneously active .recent experimental data suggest that the synaptic connections are strengthened ( long - term potentiation ) or weakened ( long - term depression ) depending on the precise relative timing between pre - synaptic and post - synaptic spikes .many current theoretical studies of learning in neural systems are based on the quasi - static hopfield model utilizing mcculloch - pitts neuronal units or simple `` integrate - and - fire '' models of neurons coupled via synapses which accumulate the pre - synaptic activity ( spikes from other neurons ) into a growing membrane potential of a neuron .the latter generates its own spike ( fires " ) once the membrane potential exceeds a certain threshold .the synaptic strength is in fact not fixed , but changes in response to external stimuli and/or inter - neural dynamics .various models based on this general picture , have been shown to yield differential hebbian learning rules .a complimentary approach to modeling the neuronal activity involves replacing neurons by periodic oscillators .indeed , rhythmic activity plays important role in many neuronal systems and functions , including central pattern generators , visual and olfactory systems , etc. .neural networks can often be described as networks of coupled oscillators , so the role of relative spike timing is played by the phases of individual oscillators ( see , for example , ) .close to the hopf bifurcation , the dynamics of the array of coupled oscillators can be described by the landau - stewart equations for complex amplitudes of the oscillators .furthermore , one can reduce the dynamics of the oscillators to the pure phase dynamics by assuming that the coupling is weak , and the amplitude of oscillations is slaved to the phase variations .in fact , the phase dynamics description may still be valid even away from the hopf bifurcation point , where the landau - stewart description is not applicable .this idea was first put forth by winfree and later developed by kuramoto and others . the original kuramoto model , in which the coupling coefficients were chosen fixed and equal , allowed for the elegant analytical treatment within the mean field approximation .it was shown that the system of globally coupled oscillators with non - identical frequencies exhibits a second - order phase transition as the coupling strength is increased above some critical value .  in subsequent work ,more complex coupling functions were considered .it was shown that depending on the choice of the connectivity matrix , synchronized states with non - trivial phase relationship among the oscillators can emerge .networks of coupled phase oscillators with appropriately tuned coupling matrix were shown to have neurocomputing properties similar to those of classical hopfield networks , with an important distinction that memorized patterns are stored in the form of relative phases of synchronized oscillators rather than in static equilibria .  in those studies ,the connectivity matrix , however complicated , was imposed externally to achieve the desired network dynamics .in fact , in order to achieve certain behavior ( for example , to memorize an image ) , in a system of neurons , coupling coefficients have to be specified. this would present significant difficulties in operating such a system at large . in biological neural networksthis task is not assigned to any central control unit , but is performed in a distributed manner via the mechanism of synaptic plasticity , i.e. long term potentiation or depression of synapses in response to the dynamics of pre - synaptic and post - synaptic neurons .external stimuli directly affect only the dynamics of individual neurons and not their connections , while the synaptic plasticity is a result of the inter - neuronal interactions .this evolution of connectivity matrix have been introduced in ref. for the landau - stewart model of coupled oscillators , however no analysis of the joint dynamics of the network of oscillators _ and _ the connectivity matrix has been presented . in this paper , we study the simplest possible model of this process , based on the generalization of the kuramoto model . in particular , we assume that the coupling coefficient for a link between two oscillators is a slow function of the phase difference between the two oscillators .  the paper is organized as follows . in section [ model ]we introduce the generalized kuramoto model with additional equations describing the slow evolution of the coupling matrix . in section [ two ] , a simple case of two coupled oscillators is investigated .we show that for large enough separation of time scales of fast and slow motions , the system is bistable : depending on initial conditions , oscillators can be either synchronized or non - synchronized . in sections [ many],[stab ] , we study the existence and stability of the family of synchronized cluster solutions in the continuum limit . in section [ memory ] , we show that the generalized kuramoto system can serve as a very simple model of learning and memory .information is stored within the synchronized cluster in the form of phase differences between oscillators .section [ conclusion ] summarizes our conclusions .let us consider the dynamics of an ensemble of coupled phase oscillators here are natural frequencies and are phases of individual oscillators , is a -periodic coupling function , and is the matrix of coupling coefficients .  in order to include the mechanism of adaptation into model ( [ phase_eq0 ] ) , we assume that an element of the coupling matrix describing interaction between two oscillators , and , is controlled by the following equation , where is a -periodic function of its argument . the particular form of the coupling function and adaptation function can be derived from the underlying equations describing the dynamics of oscillators through the reduction to the phase description .following kuramoto , we choose the simplest possible periodic function .furthermore , we choose the adaptation function .this choice implies that the coupling coefficient grows fastest for two oscillators which are in - phase and decays fastest for out - of - phase oscillators .there are certain indication in the biological literature ( see ref. ) that the maximal ltp and ltd correspond to small positive and negative phase shifts between pre - synaptic and post - synaptic neuronal oscillations , but we are going to ignore this complication in this model for the sake of simplicity . with the chosen functions ,our model becomes   for small , the dynamics of the coupling coefficients is slow , and one can expect that for two oscillators which are non - synchronized , the driving term is oscillating around zero , and the resultant coupling coefficient is also oscillating around zero and is small . on the other hand , a pair of synchronized oscillators produces a constant non - zero driving for the corresponding coupling coefficient , and therefore for large enough , the connection between the two will be strengthened .thus , we anticipate a multi - stable behavior , when a group of oscillators can either be stably synchronized or non - synchronized depending on the initial conditions for their coupling coefficient .to illustrate this point , we first consider a simple case of two oscillators symmetrically coupled through a single link .in this case model ( [ phase_eq]),([k_eq ] ) reduces to a set of two coupled equations , here , , and . by introducing new variables , system ( [ two_osc ] )is simplified to   the nullclines of this system and intersect at large enough . in this case , within each period of the relative phase , there are four intersections corresponding to two stable and two unstable fixed points ( see figures [ phase_plane],b , c . for small , the dynamics of the system can be separated into fast and slow motions . if initial value , the phase variable rapidly approaches the quasi - static value corresponding to the nullcline , without any significant change of .then , depending on whether is greater or smaller than the value corresponding to the unstable fixed point , the solution either slowly approaches the stable fixed point ( synchronized regime ) , or it reaches the minimum of the nullcline and then branches off to infinity along the fast trajectory . the same occurs if .this corresponds to the regime of non - synchronized oscillations . for larger values of , the dynamicscan no longer be reduced to fast and slow motions . moreover , for any fixed , there exists some critical value of such that at there are no unbounded solutions corresponding to the non - synchronized motion . on the other hand ,if , there are no fixed points corresponding to the synchronized solutions , and the oscillators are desynchronized for any values of .figure [ phase_plane]b , c shows the structure of the phase plane for system ( [ two_osc1 ] ) for and , respectively . figure [ alpha_eps ] shows the dependence of vs. .as expected , diverges as .this line together with vertical line divides the parameter plane into three regions . for , there can only be non - synchronized solutions ( fig.[phase_plane],a ) , for there are only synchronized solutions ( fig.[phase_plane],b ) , and for , there is a bistable state when both desynchronized and synchronized regimes are stable ( fig.[phase_plane],c ) .the results of this section can be applied to a more generic case of many coupled oscillators .indeed , as and/or is increased , the spontaneous clustering usually begins from synchronization of pairs of neighboring oscillators . for this process, the coupling between these oscillators and other oscillators can be neglected .symbols in fig .[ alpha_eps ] show the results of numerical experiments with a population of 50 oscillators uniformly distributed within a range starting from small randomized initial coupling .the bifurcation to random parings of oscillators which precedes the cluster formation , occurs close to the theoretical line predicted for a simple case of two coupled oscillators .in this section we return to the dynamics of the original model ( [ phase_eq]),([k_eq ] ) .as was pointed out in section [ model ] , for small , in the asymptotic regime the coupling coefficients are either non - stationary but small ( for two oscillators operating at different frequencies ) or fixed for two oscillators which are synchronized .so , in the limit , we can neglect all for non - synchronized oscillators , and define for synchronized oscillators . substituting this into eq.([phase_eq ] ) , we obtain for oscillators within a cluster , .\label{ph_eq1}\ ] ] where unlike ref. summation only applies to oscillators within the same cluster , and for oscillators outside synchronized cluster .it is easy to see that eq.([ph_eq1 ] ) is formally equivalent to the kuramoto model for the double phases .one can investigate this model in the mean - field limit using the order parameter method . in this case, the complex order parameter reads where , is the frequency of the synchronized cluster , is the phase distribution of the oscillators in the population which belong to the cluster .the phases of the oscillators inside the cluster are given by and the phase distribution can be easily expressed via the frequency distribution of the synchronized oscillators , where determine size and position of the synchronized cluster , ( ) . substituting ( [ distr ] ) into eq.([op ] ) , we obtain the equation for the magnitude of the order parameter  the frequency spread inside the synchronized cluster is determined by formula which follows directly from eqs.([phases ] ) .thus , this system exhibit degeneracy so that a variety of synchronized states can be formed depending on initial conditions .it is particularly evident for the uniform distribution of oscillator frequencies within frequency interval , . in this case , the cluster size drops out , the limits of integration are symmetric , , and we arrive at which reduces to an algebraic equation for phase width of the cluster , the phase width of the cluster depends only on and does not depend on the cluster size .this dependence is shown in fig.[clusterwidth ] . for large , . at , .the frequency spread inside the synchronized cluster is equal to ( cf .( [ omega_c ] ) ) . at , there can be no stationary synchronized clusters .these findings are confirmed by direct numerical simulations of eqs.([phase_eq]),([k_eq ] ) . in fig.[clusterwidth ] the phase width of the cluster is shown for several values of , and in fig.[unstablecluster ] the evolution of an unstable cluster at is shown .the parameters of simulations were , and the initial cluster width was chosen .as we have seen in the previous section , the order parameter which characterizes the cluster size , can take any value from 0 to ( for the uniform frequency distribution the latter corresponds to the whole population being synchronized , ) .however , this is true only for small enough . just as in the case of two coupled oscillators ( sec.[two ] ) , at sufficiently large , the multistability disappears , and the only stable state at is the one involving the whole population of oscillators , . in order to determine , we analyze the following problem .suppose we have a synchronized cluster oscillating at frequency and characterized by the order parameter .let us consider a single oscillator with frequency adjacent to the cluster ( ) , and study its dynamics neglecting its interaction with all other oscillators and the the feedback influence of the oscillator on the cluster .clearly , since the frequency difference between this oscillator and the cluster is smaller than that for any other non - synchronized oscillator , the adjacent oscillator will be the first to get entrained at .the equation for the relative phase of this oscillator reads and the equation for the coupling coefficient between the oscillator and the -th member of the cluster , the summation in eq.([ph_eq2 ] ) is carried only over synchronized oscillators .the phases of the synchronized oscillators are determined by eq.([phases ] ) .  in the continuum limit for the uniform frequency distribution ,these equations become eq.([ph_eq3 ] ) can be rewritten in the form where equations for can be obtained using eq.([k_eq3 ] ) , where .the set of equations ( [ ph_eq4]),([p_eq]),([q_eq ] ) exhibits the dynamics similar to eqs.([two_osc1 ] ) for two coupled oscillators . at ,the only attractor of the system is the solution with periodic and and unbounded phase which corresponds to `` non - entrainment '' of the oscillator by the cluster . for the phase variable taken _modulo _ , this solution corresponds to a limit cycle encircling the phase cylinder . at larger ,two fixed points appear , one of which ( stable ) corresponds to the entrainment of the oscillator by the cluster . at ,the limit cycle disappears , and the entrained state is the only stable state of the system .thus , the critical line separates the region of cluster stability with respect to entraining additional oscillators .this line is shown in figure [ epsilon0 ] .this line should be compared with the critical line for pairing of two neighboring oscillators ( fig.[alpha_eps ] ) .note that we have to re - scale and back into original and using , . for large , this line lies below , which indicates that the random pairing outside the cluster occurs before the main cluster loses its stability .this conclusion agrees with our numerical simulations .information can be stored within the cluster in the form of relative phases of the oscillators in the synchronized state .initial learning can be accomplished via driving a group of oscillators by external signals with identical frequency and magnitude , but with different phases , if , every oscillator in the group will be entrained to the external frequency .so , the external driving signals force the oscillators to oscillate in synchrony and , via slow synaptic dynamics eq.([k_eq1 ] ) , form a tightly coupled cluster . if the phases are taken to be 0 or , the relative phases within the cluster will be also close to or , and so the coupling coefficients will approach . after the learningis completed after time , the external signal is disconnected ( ) .if the external frequency , the mean frequency of oscillators within the cluster , the cluster remains synchronized at the same frequency , moreover , the phase relations among the oscillators are effectively preserved .even significant random fluctuations affecting the dynamics of individual oscillators , do not change the robust structure of the synchronized cluster , see fig.[clusterphase ] .  for small ,after a cluster is formed , it becomes very stable with respect to external noise .furthermore , it remains stable with respect to random variations of their natural frequencies .we let the cluster form similarly to described above , and then at some time , we perturbed the natural frequencies of all oscillators , , where are i.i.d .gaussian random variable with variance 1 .we can quantify the robustness of the cluster by measuring the correlation between the synchronized cluster structure before and after the random fluctuations of natural frequencies have been applied .the correlation was calculated using the formula where is the binary variable describing the state of the oscillator before(after ) perturbing the natural frequencies : if the oscillator is entrained in the main cluster and otherwise .symbols and denote boolean addition and multiplication , respectively .it is easy to see that when cluster remains unchanged , and if all oscillators leave the cluster .figure [ correlation ] shows correlation as a function of the perturbation magnitude for several values of .as can be seen in the figure , the cluster becomes more robust with increase of .in summary , we have demonstrated that a generalized kuramoto model with additional equations describing slow dynamics of the coupling matrix can be used to elucidate the underlying mechanism of synaptic plasticity .the slow dynamics leads to the multi - stability : synchronized clusters of different sizes and with different phase relationships among oscillators can be stabilized .the phase differences among the oscillators can be used as a way of storing and retrieving the information in this system .one natural limitation of the reduced phase description of the oscillators is that all oscillators are assumed to be in the excited ( `` firing '' ) state .however , in natural systems ( such as biological neural networks ) , some of the neurons may be in quiescent ( `` non - firing '' ) regime .a more general description of this system should incorporate equations for complex amplitudes of oscillators similar to ref. .our results may have important biological implications .in particular , they suggest that there are fundamental reasons for the wide separation of time scales of fast neuronal dynamics ( e.g. membrane potential oscillations ) and slow synaptic variability. they also present a very simple ( possibly , the simplest ) model of the _ synaptic reentry reinforcement _ which is believed to be the foundation of the long - term memory stability .e. shimizu , y .- p .tang , c. rampon , and j. z. tsien , science , * 290 * , 1170 - 1174 ( 2000 ) . l.f.abbott and s.b.nelson , nature neuroscience suppl . , * 3*m 1178 ( 2000 ) .hebb , the organization of behavior , wiley , new york , 1949 .h. markram , j. lbke , m. frotscher , b. sakmann , science , * 275 * , 213 - 215 ( 1997 ) .l.i.zhang , h.w.tao , c.e.holt , w.a.harris , m.poo , nature , * 395 * , 37 ( 1998 ) .j.j.hopfield , proc ., * 79 * , 2554 ( 1982 ) .h. tuckwell , _ introduction to theoretical neurobiology ._ cambridge , 1988 .roberts , phys .e , * 62 * , 4077 ( 2000 ) .h.d.i.abarbanel , r. huerta , m.i.rabinovich , in preparation .c.m.gray , j.comput .neurosci . , * 1 * , 11 ( 1994 ) .h.sompolinsky , d.golomb , and d.kleinfeld , phys .a , * 43 * , 6990 ( 1991 ) .t.aoyagi , phys .* 74 * , 4075 ( 1995 ) .f.c . hoppensteadt and e.m.izhikevich , biol .* 75 * , 129 ( 1996 ) .f.c . hoppensteadt and e.m.izhikevich , ieee trans .neural networks , * 11 * , 734 ( 2000 ) .a.t.winfree , j. theor .* 16 * , 15 ( 1967 ) .y.kuramoto , _ chemical oscillations , waves , and turbulence _ ( springer , new york , 1984 ) .see review by s.h.strogatz , physica d * 143 * , 1 ( 2000 ) .m.k.yueng and s.h .strogatz , phys .* 82 * , 648 .( 1999 ) f.c .hoppensteadt and e.m.izhikevich , phys .lett . , * 82 * , 2983 ( 1999 ) .

Abstract: a generalized kuramoto model of coupled phase oscillators with slowly varying coupling matrix is studied . the dynamics of the coupling coefficients is driven by the phase difference of pairs of oscillators in such a way that the coupling strengthens for synchronized oscillators and weakens for non - synchronized pairs . the system possesses a family of stable solutions corresponding to synchronized clusters of different sizes . a particular cluster can be formed by applying external driving at a given frequency to a group of oscillators . once established , the synchronized state is robust against noise and small variations in natural frequencies . the phase differences between oscillators within the synchronized cluster can be used for information storage and retrieval .