the development of social network has a great impact on our lifestyle , from making friends to dating , from working to shopping .they become more essential as we are increasing our dependence on them to gather information . compared with search engines which are based on isolated queries , collecting information through leveraging the individual specialties in social networkslead us to useful websites from experts in disparate fields , and thus increase both the quality and the diversity of the acquired information .thus , by the same token , influential individuals can also be used to spread information .the key to the success is to identify the most influential spreaders in the network .nevertheless , it is difficult as there are usually just a few users capable of propagating a news to a large number of users .for example , while socially significant users are rare in the tweeter network , their messages and blogs can spread quickly throughout the whole network .although social networks are powerful for propagating information , their application for this purpose is limited , partially because a way to identify the optimal spreaders is absent .nevertheless , simple methods have been proposed .for instance , degree centrality " suggests that nodes with higher degree are more influential than the others . on the other hand ,the location of a node in a network and the influence of its neighbors are also considered important .for instance , a node with a small number of highly influential neighbors located at the center of the network may be more influential than a node having a larger number of less influential neighbors .kitsak et al . thus proposed a coarse - grained method to use the -core decomposition to quantify the influence of a node , based on the assumption that news initiated at nodes in higher shells are likely to spread more extensively .some distance - based global metrics such as betweenness and closeness are suggested which can lead to extensive propagation , but due to the high computational complexity , they are not practical for large - scale social networks .other centralities such as localrank were also suggested .the above simple but sub - optimal protocols have been applied to social media such as qq , bbs and blog to find the key spreaders who can trigger the tipping point " in social marketing to promote commercial products .specifically , if one can convince a set of influential users to adopt a new product , one may induce a large cascade of purchases as these initial buyers propagate their compliment of the product along the network . unlike the forementioned methods which identify a set of independent spreaders according to their centralities , our goal is to find a set of coordinated individuals such that their combined impact is greatest , leading to much more extensive propagation of information .nevertheless , identifying the optimal group of spreaders is indeed a computationally difficult task .  in this paper, we utilize the similarities between percolation and information propagation to identify a group of influential spreaders . by removing edges at random until percolation ceases ,individual isolated clusters are formed . due to the correspondence between percolation and information transmission ,the emergence of such clusters imply that news can be effectively propagated within the clusters but not across the clusters .initiating a news on the most influential user in each cluster is thus an effective way to distribute the news within the cluster .since such process is static and requires much less computation power than the dynamics spreading of news , a lot of percolated states can be generated to give a more accurate result on the segmentation of social clusters as well as their corresponding influential spreaders .  by testing our protocol on facebook and enron email network ,we show that in addition to a lower computational efficiency , our protocol outperforms other simple heuristics based on local and global centrality in terms of propagation coverage and coverage redundancy of the selected spreaders .this is consistent with the old saying that the power of a typical group exceeds that of a single most competent individual .moreover , we find that the average degree of the users selected by our method is lower , which implies a lower cost in identifying the spreaders when compared to the other methods .we also identify the different characteristics of spreaders who are most effective to promote niche or popular items in order to maximize the coverage .all these results lead to insights into the design of viral marketing strategies and a new paradigm for information propagation .spreading dynamics with the involvement of human can be mainly classified into two classes : one is the spreading of infectious diseases which requires physical contacts , and the other is the spreading of information , including opinions and rumors where physical contacts are not required . due to the similarity between epidemic and information spreading ,well - established models of epidemic models are widely used to describe the propagation of information .  in particular , the susceptible - infected - recovered ( sir ) model is one of the representatives .specifically , a susceptible person ( s ) in the model is analogous to an individual who is not aware of the information .an infected person ( i ) is analogous to an individual who is aware of the information and will pass it to his / her neighbors .a recovered person ( r ) is analogous to an individual who loses his / her interest and will never pass the information again .newman studied in detail the relation between the static properties of the sir model and bond percolation phenomenon on networks and remarked that the sir model with transmissibility is equivalent to a bond percolation model with bond occupation probability on the network ( see method and materials 4.5 ) .  our method is then devised in relation to the bond percolation model as follows .given an undirected network where represents the set of nodes ( i.e. , users in social networks ) and represents the set of edges ( i.e. , connection in terms of communication , friendship or other kinds of interactions ) , all edges are first removed and each individual edge is then recovered with a probability , i.e. all links are removed when . as increases from ,more links are recovered and clusters start to form and merge with each other. we will call this state the _ pre - percolated state_. for a network containing node , a giant component of size emerges only when is larger than a critical threshold , which is called percolation . in the context of information propagation , since an edge between two nodes appears with a probability , the value can be considered as the transmissibility of an information from one node to another .  to find the influential group, we have to find the most influential spreaders with a given value of .assume that there are percolation clusters after one realization of link recovery , and denote by the size of cluster , .we introduce a tunable parameter , which is usually equal to or larger than .if , we choose the top- largest clusters and assign one score to the largest degree node in each cluster .if there are many nodes with the largest degree , we assign the score to a random one among them .if , we first choose the top- node in each cluster , and the rest nodes are chosen to be those with the second largest degree respectively from the top- largest clusters . if , we will choose the next largest degree nodes in each cluster following the same selection rules .after times of different trials of link recovery , all nodes are ranked according to their scores in a descending order and those nodes with the highest scores are suggested to be the set of initial spreaders . for the sake of simplicity, we set and have tested and found that the results are not sensitive to .the dependence of are shown in the supplementary information ( si ) sec .s6 .  in other words, our suggested method draws analogy with percolation to identify individual social clusters in the network where news can be effectively propagated within the clusters but not across the clusters .these isolated clusters in the pre - percolated state thus have a direct correspondence to the propagation coverage when one spreads the news from an initial spreader in each of the clusters .our rationale is different from most other methods which usually identify a group of influential spreaders for the network as a whole .in addition , such a set of well distributed spreaders also enjoy a reduced redundancy when compared to a set of un - coordinated spreaders .these differences make our method unique compared to the other methods .to quantify the performance of our method , we examine the spreadability , i.e. the propagation coverage of a news from a set of selected spreaders , by our method as well as other methods .we will use the sir model to mimic the spreading of news , and the spreadability is defined as the ratio of recovered nodes to the total number of nodes ( i.e. , the size of outbreak to ) .we remark that the transmissibility adopted in the sir model is the same as the probability used to recover edges to identify the clusters in the percolated states . as a result , for a single spreader, the ultimate size of the sir outbreak triggered by this spreader is precisely the size of the percolation cluster that it belongs to .likewise , the ultimate size of the sir outbreak triggered by a group of spreaders in distinct percolation clusters is the sum of the size of the clusters that these nodes belong to .for example , if we measure the coverage of three selected nodes on the network with nodes , and if the first two nodes belong to the cluster , and the third one is in the cluster . for each of the single nodes ,the coverage of node 1,2 and 3 are respectively , and , while for the whole group , the spreadability of the three nodes is .we first apply our method on the facebook network with 59691 nodes .figure 1a shows the coverage obtained from 4000 initial spreaders chosen by our percolation method , compared with a set of 4000 spreaders identified by three other methods , namely the degree centrality , the -shell decomposition and the betweeness centrality ( see * methods and materials * for the definition of each of these methods ; comparisons with other centrality measures can be found in sec .s2 of the si ) .percolation method yields the highest spreadability for an arbitrary transmissibility .figure 1b shows the degree distribution of the 4000 spreaders identified by the percolation method .when ,the percolation method yields isolated clusters ( see fig .1d ) of similar size , and since the set of the selected spreaders come from different clusters , and a wide range of degree is found among the spreaders ( see fig .1b ) . in this case , the percolation method is more likely to choose high - degree nodes ( see fig .s5b in the si , where the red stars represent the degree distribution of the 4000 selected nodes when ) .when , the distribution will become narrower as increases ( see the blue squares in fig .s5b of the si ) . in this case, the percolation method prefers low - degree spreaders . the average original degree ( i.e. degree in the original network before edge removal ) of the 4000 spreaders selected by the percolation method when is higher than that of the nodes selected when .this implies that if we want to promote and advertise a new niche product which is difficult to get accepted , one can draw analogy with the case of small transmissibility where high - degree initial spreaders are preferred .on the other hand , for popular items which are easy to be accepted , one can draw analogy with the case of large and low - degree initial spreaders are preferred .we then examine the cost of identifying the initial spreaders . by assuming that the direct influence of a user is equal to the number of its nearest neighbors ( i.e. , its degree ) , while the difficulties of finding a user with degree is proportional to , the cost to find a spreader assumed to be .figure 1c shows the dependence of the average cost to find the 4000 spreaders under the parameter , i.e. .the cost decreases abruptly at the critical point , indicating a phenomenon resembling phase transition .it means that when increases just beyond , the cost can be reduced substantially .  besides the spreadability and the cost, we also examine the redundancy in coverage which quantifies the efficiency of the propagation . specifically , the redundancy of a node is defined as the number of initial spreaders who has the potential to infect node .a method is inefficient if the initial chosen spreaders pass the same information to the same group more than once .averaging the redundancy over all the infected nodes , we obtain the redundancy of the set of initial spreaders .figure 1e compares the spreading redundancy of our method with the three other methods ( comparisons with other centrality measures can be found in sec .s3 of the si ) .highest redundancy is found in the methods of -shell and degree centrality , followed by betweenness centrality .our percolation method has the lowest redundancy among the four methods , since the spreaders identified by this method are usually located in different regions of the original network .we also checked the enron e - mail network and similar results with facebook network are obtained ( see supplementary sec.ii figure s1 ) .  to further examine the spreadability, we applied the four methods to identify four initial spreaders on a generated network with four clear communities .as shown in fig .2 , the four spreaders identified by the percolation method are very likely to be found in different communities , with one spreader in each community .for the other methods , there are high probabilities that all or some of the initial spreaders are in the same communities .these results are easy to understand as our method relies on the segmentation of the network into isolated clusters to identify the spreaders . in the present case ,the network is likely to separate into the four communities and thus one spreader is found in each community .-shell ( red dash line ) and betweeness centrality ( dots ) .( b ) the degree distribution s of the 4000 initial spreaders selected by the percolation method given different transmissibility .the color indicates the frequency , blue to red corresponds to low to high .( c ) the cost to identify such a group of spreaders .( d ) the size of the second giant component , , after percolation with different transmissibility . the largest value of  is obtained at the critical point .( e ) the coverage redundancy of the four methods.,width=604 ]  .the percentage of the different distribution of the four spreaders in a model network with four communities .for instance , the four spreaders identified by our methods are found in four different communities in 93.9 of the realizations .the procedure to construct the networks is shown in method and materials .the results are obtained based on 1000 realizations .the critical point of the network is .we have set in our percolation method . [cols="^,^,^,^,^,^",options="header " , ]   [ table_cummunity ]  most of the other methods always lead to the same set of spreaders . in our percolation method ,different set of spreaders may be generated from different realizations , especially for large .figure [ overlap ] ( a ) shows the number of initial spreaders which are common among different realizations of the percolation method applied on the facebook network . it is clear that when increases , the number of common spreaders decreases , indicating that the solutions become more and more diverse .this result has practical significance , especially when some of the initial spreaders are offline , we can use the next best candidate as a back - up spreader without losing spreadability .on the other hand , fig .[ overlap ] ( b ) shows the entropy of the obtained solution , i.e. the logarithm of the number of different identified spreaders .compared with the other three methods , percolation method provides a higher flexibility in the choice of spreaders .+  in order to further examine the difference between our method and the other methods , fig .[ overlap4 ] shows the number of identified spreaders which are common between the percolation method and the other methods ( comparisons with some other methods are found in sec .siii fig s4 of the si ) .the overlap between the percolation method and the degree centrality method reaches the highest value ( ) at the critical point and then sharply decreases to less than when .it is because when increases , most of the high - degree nodes are replaced by nodes with lower degree , and there are a lot of sets of identified spreaders generated from the different realizations as we have discussed in fig .[ overlap ] .what is more impressive is that by increasing the value of the cost can sharply decrease without losing spreadability and substantially increasing coverage redundancy ( see fig [ example](c ) and [ example](e ) ) .we show in fig .5 the relation between the spreadability and the cost .the percolation method is the most cost effective method in terms of spreadability .four cases are presented , namely , , and .clearly , with the same cost , the percolation method lead to a higher spreadability than the methods of -shell and degree centrality .although the cost for using betweenness is low , its spreadability is very limited and become saturated at small cost .the percolation method has the highest saturated value of spreadability .   for the percolation method.,width=529 ]   in the percolation method .here we present a typical case as examples . for a given , we select 4000 top - ranked nodes according to the different methods .for example , to obtain the results of the percolation method ( i.e. blue curve ) , we draw the total cost of the top- ( ) nodes as -axis value and their corresponding spreadability ( as a group ) as -axis value . , title="fig:",width=264 ]  in the percolation method . here we present a typical case as examples .for a given , we select 4000 top - ranked nodes according to the different methods . for example , to obtain the results of the percolation method ( i.e. blue curve ) , we draw the total cost of the top- ( ) nodes as -axis value and their corresponding spreadability ( as a group ) as -axis value . , title="fig:",width=264 ] +  +  in the percolation method . herewe present a typical case as examples . for a given , we select 4000 top - ranked nodes according to the different methods . for example , to obtain the results of the percolation method ( i.e. blue curve ) , we draw the total cost of the top- ( ) nodes as -axis value and their corresponding spreadability ( as a group ) as -axis value . , title="fig:",width=264 ]  in the percolation method . herewe present a typical case as examples . for a given , we select 4000 top - ranked nodes according to the different methods .for example , to obtain the results of the percolation method ( i.e. blue curve ) , we draw the total cost of the top- ( ) nodes as -axis value and their corresponding spreadability ( as a group ) as -axis value . , title="fig:",width=264 ] +  +as we can see , social networks constitute a new platform to propagate information . unlike the usual practice where the networks are used by uncoordinated individuals to share their own message , intended spreading of informationcan indeed be implemented via the networks . to measure its performance, one can measure the coverage , the redundancy in propagation , and the cost in identifying appropriate initial spreaders . yet these measures of performance are largely dependent on the choice of users who start the propagation , and there is not a single protocol which achieves optimality in all these dimensions .these difficulties of identifying influential spreaders makes information propagation via social networks remain in its immature stage .  to tackle the challenge , we draw an analogy between the percolation process and information propagation to develop a protocol which gives rise to a low - cost , minimally redundant set of initial spreaders leading to a large coverage . our protocol was tested on the facebook network , where favorable results over all the tested centrality - based methods were obtained .when compared to these conventional methods which identify a set of un - coordinated spreaders , the spreaders identified by our protocol are evenly distributed within the network which greatly increases the propagation coverage and reduces its redundancy .such coordination of spreaders is essential and can only be obtained using the suggested percolation procedures .the success of this method is not just a coincidence , but it makes the best use of the similarities between percolation and the process of information propagation . by removing edges at random until percolation ceases , we identify individual isolated clusters where news can be effectively propagated within the clusters but not across the clusters .specific spreaders at the center of these clusters are then identified to be the influential initial spreaders in the original network . by initiating news propagating from this set of spreaders ,coverage is increased and redundancy is reduced compared to the conventional centrality methods .percolation is thus at the center of our propagation protocol instead of a mere analogy .the remaining question is practicality . as we have discussed , the computational complexity of our protocol is , which is a favorable characteristics for applications on practical systems as its complexity scales linearly with the system size .once the set of important initial spreaders is identified , an coordinator just has to connect to these users and pass the news to them , and information will then propagate quickly throughout the network . of course , a lot of details and practical difficulties are omitted in this simple description , but our results have lead to insights into a completely new paradigm of information propagation .further research along this line may revolutionize our way of spreading and gathering information in the near future .to identify the most influential spreaders , various centrality measures have been proposed .the first method by which we compare our result with is degree centrality .degree centrality is a straightforward and efficient metric .it assumes that a node with more nearest neighbors has a higher influence .however , node degree can only reflect its direct influence but not the indirect influence triggered by its nearest neighbors .for example , a node of small degrees , but with a few highly influential neighbors may be more influential than a node having a larger number of less influential neighbors .the second method we used for comparison is the -shell decomposition .recent research shows that the location of a node in a network may play a more important role than its degree .a node located in the center of the network is more influential than a node having a larger number of less influential neighbors .similar to this rationale , kitsak et al . proposed a coarse - grained method by using the method of k - core decomposition to quantify the influence of a node based on the assumption that nodes in the same shell have similar influence , and nodes in higher - level shells are likely to infect more nodes .  in the last method, we employ global information to identify the influential spreaders .specifically , betweenness is one of the most popular geodesic - path - based ranking measures .it is defined as the fraction of shortest paths between all node pairs that pass through the node of interest .betweenness is , in some sense , a measure of the influence of a node in terms of its role in spreading information . for a network with nodes and edges ,the betweenness centrality of node , denoted by is  where is the number of shortest paths between nodes and , and denotes the number of shortest paths between nodes and which pass through node .given a network , there are four steps to find the influential spreaders by the percolation method .firstly , all the edges are first removed and then recovered with a probability ; we then obtain a new network .the required computational complexity is .secondly , we find the strongly connected components of using tarjan s algorithm which has a complexity of .thirdly , we select one node with the highest degree in each of the largest components and assign one score to the selected nodes .this complexity for the procedures is . repeating the above three steps for different realizations, we rank the nodes according to their scores in descending order , and the top- nodes are chosen to be the most influential spreaders .the different realizations of the percolation process can be computed in parallel and the complexity of each implementation is .considering , then the complexity is  ] , i.e. the complexity of our method grows linearly with system size .there are three steps to generate a network with community structures . in our experiment, we consider a network with nodes which has four communities each of which contains 500 nodes .first , we generate a random network of size 500 and with node degrees distributed in power - law with exponent 2.2 using the configuration model .the minimum degree is 1 and the maximum degree is  .second , we repeat the above procedures to generate independently the other three networks . finally , for each pair of sub - networks we randomly selected a fraction of node pairs to connect them .datasets we used are described in sec .i of the si and the statistical features of the real networks are summarised in table s1 .susceptible - infected - recovered ( sir ) model is usually used to mimic the spreading processes of disease .individuals in this model are classified in three states : susceptible ( , does not carry the disease and will not infect others but can be infected ) , infected ( , carry the disease and can infect others ) , recovered ( , either dead or recovered from the disease and immune to further infection ) .the simulation runs in discrete time steps . at each time step ,infective individuals transmit the disease to his or her neighbors with probability and will recover with probability .then the sir transmissibility is .the process stops when there is no infected node anymore .the sir model can be mapped to a bond percolation model where each link exists with a probability equals to the sir transmissibility  . after removing the other edges ,a number of clusters are formed .it is clear that the ultimate size of the sir epidemic outbreak is triggered by a single initially infected node , which is precisely the size of the percolation cluster that the initial node belongs to .apparently , the nodes in the same cluster are expected to have the same coverage . a review article on epidemic processes in complex networkscan be found in ref .this work is partially supported by the nsfc grant nos .61203156 , 11205042 , and the fundamental research funds for the central universities gran no .ll acknowledges research start - up fund of hangzhou normal university under grant pe13002004039 and the eu fp7 grant 611272 ( project growthcom ) .chy acknowledges the internal research grant rg 71/2013 - 2014r of the hong kong institute of education .jianshu weng , ee - peng lim , jing jiang , and qi he .twitterrank : finding topic - sensitive in uential twitterers . in proceedings of the third acm international conference on web search and data mining , pages 261270 .acm , 2010 .yanqing hu , shlomo havlin , and hernn a makse .conditions for viral in uence spreading through multiplex correlated social networks .physical review x , 4(2):021031 , 2014 .stanley wasserman .social network analysis : methods and applications , volume 8 .cambridge university press , 1994 .maksim kitsak , lazaros k. gallos , shlomo havlin , fredrik liljeros , lev muchnik , h. eugene stanley , and hernan a. makse. identification of in uential spreaders in complex networks .nat phys , 6(11):888893 , 11 2010 .  linton c freeman . a set of measures of centrality based on betweenness .sociometry , pages 3541 , 1977 .gert sabidussi .the centrality index of a graph .psychometrika , 31(4):581603 , 1966 .duanbing chen , linyuan l , ming - sheng shang , yi - cheng zhang , and tao zhou .identifying influential nodes in complex networks .physica a : statistical mechanics and its applications , 391(4):17771787 , 2012 .david kempe , jon kleinberg , and va tardos . maximizing the spread of in uence through a social network . in proceedings ofthe ninth acm sigkdd international conference on knowledge discovery and data mining , pages 137146 .acm , 2003 .romualdo pastor - satorras , claudio castellano , piet van mieghem , and alessandro vespignani .epidemic processes in complex networks .arxiv preprint arxiv:1408.2701 , 2014 .mark ej newman .spread of epidemic disease on networks .physical review e , 66(1):016128 , 2002 .linyuan l , duan - bing chen , and tao zhou .the small world yields the most effective information spreading .new journal of physics , 13(12):123005 , 2011 .aidan sudbury .the proportion of the population never hearing a rumour .journal of applied probability , pages 443446 , 1985 .damian h zanette .critical behavior of propagation on small - world networks .physical review e , 64(5):050901 , 2001 . damian h zanette .dynamics of rumor propagation on small - world networks .physical review e , 65(4):041908 , 2002 .zonghua liu , ying - cheng lai , and nong ye .propagation and immunization of infection on general networks with both homogeneous and heterogeneous components .physical review e , 67(3):031911 , 2003 .yamir moreno , maziar nekovee , and amalio f pacheco .dynamics of rumor spreading in complex networks .physical review e , 69(6):066130 , 2004 .mark ej newman , steven h strogatz , and duncan j watts .random graphs with arbitrary degree distributions and their applications .physical review e , 64(2):026118 , 2001 .roni parshani , sergey v buldyrev , and shlomo havlin .critical effect of dependency groups on the function of networks .proceedings of the national academy of sciences , 108(3):1007 - 1010,2011 .roger guimer , albert diaz - guilera , fernando vega - redondo , antonio cabrales , and alex arenas .optimal network topologies for local search with congestion .physical review letters,89(24):248701 , 2002 .gang yan , tao zhou , bo hu , zhong - qian fu , and bing - hong wang .effcient routing on complex networks .physical review e , 73(4):046108 , 2006 .linton c freeman .centrality in social networks conceptual clarification .social networks , 1(3):215239 , 1979 .robert tarjan .depth - first search and linear graph algorithms .siam journal on computing , 1(2):146160 , 1972 .michele catanzaro , marin bogu , and romualdo pastor - satorras .generation of uncorrelated random scale - free networks .physical review e , 71(2):027103 , 2005 . s. n. dorogovtsev , j. f. f. mendes , and a. n. samukhin .size - dependent degree distribution of a scale - free growing network .e , 63:062101 , may 2001 .roy m anderson , rm may , and b anderson .infectious diseases of humans : dynamics and control .australian journal of public health , 16:208212 , 1992 .

Abstract: social networks constitute a new platform for information propagation , but its success is crucially dependent on the choice of spreaders who initiate the spreading of information . in this paper , we remove edges in a network at random and the network segments into isolated clusters . the most important nodes in each cluster then form a group of influential spreaders , such that news propagating from them would lead to an extensive coverage and minimal redundancy . the method well utilizes the similarities between the pre - percolated state and the coverage of information propagation in each social cluster to obtain a set of distributed and coordinated spreaders . our tests on the facebook networks show that this method outperforms conventional methods based on centrality . the suggested way of identifying influential spreaders thus sheds light on a new paradigm of information propagation on social networks .


message - passing algorithms , like belief propagation and its variants and generalizations , have been shown empirically to be very effective in solving many instances of hard / computationally intensive problems in a wide range of fields .these algorithms were originally designed for exact inference ( i.e. calculation of marginals / max - marginals ) in tree - structured probability distributions .their application to general graphs involves replicating their iterative local update rules on the general graph . in this case however , there are no guarantees of either convergence or correctness in general .  understanding and characterizingthe performance of message - passing algorithms in general graphs remains an active research area . show correctness for graphs with at most one cycle . show that for gaussian problems the sum - product algorithm finds the correct means upon convergence , but does not always find the correct variances . show asymptotic correctness for random graphs associated with decoding . shows that if max - product converges , then it is optimal in a relatively large `` local '' neighborhood .in this paper we consider the problem of using max - product to find the maximum weight matching in an arbitrary graph with arbitrary edge weights .this problem can be formulated as an integer program , which has a natural lp relaxation . in this paperwe prove the following  1 .if the lp relaxation is tight , then max - product always converges , and that too to the correct answer .if the lp relaxation is loose , then max - product does not converge .bayati , shah and sharma were the first to investigate max - product for the weighted matching problem .they showed that if the graph is bipartite then max - product always converges to the correct answer .recently , this result has been extended to -matchings on bipartite graphs . since the lp relaxation is always tight for bipartite graphs ,the first part of our results recover their results and can be viewed as the correct generalization to arbitrary graphs , since in this case the tightness is a function of structure as well as weights .we would like to point out three features of our work :  1 .it provides a _ necessary and sufficient _condition for convergnce of max - product in arbitrary problem instances .there are very few non - trivial classes of problems for which there is such a tight characterization of message - passing performance .the characterization is _ data dependent _ : it is decided based not only on the graph structure but also on the weights of the particular instance .3 .  tightness of lp relaxations is well - studied for broad classes of problems , making this chracterization promising in terms of both understanding and development of new algorithms .relations , similarities and comparisons between max - product and linear programming have been used / mentioned by several authors , and an exact characterization of this relationship in general remains an interesting endeavor .in particular , it would be interesting to investigate the implications of these results as regards elucidating the relationship between iterative decoding of channel codes and lp decoding .a _ matching _ in a graph is a set of edges such that no two edges in the set are incident on the same node .given a graph , with non - negative weights on the edges , the _ weighted matching problem _ is to find the matching whose edges have the highest total weight . in this paperwe find it convenient to refer to edges both as and as , where .weighted matching can be written as the following integer program ( ip ) :    the lp relaxation of the above problem is to replace the constraint with the constraint ._ this relaxation is in general not tight _ , i.e. there might exist non - integer solutions with strictly higher value than any integral solution .it is known however that the lp relaxation is _ always _ tight for bipartite graphs : no matter what the edge weights , the bipartite - ness ensures tightness of the lp relaxation . if a graph is not bipartite , the tightness of the lp relaxation will depend on the edge weights : the same graph may have tightness for one set of weights and looseness for another set .the dual of the above linear program is the _ vertex cover _ problem : minimize the total of the weights that need to be placed on nodes so as to `` cover '' the edge weights : ( dp )    [ lem : lp_tight ] when the lp relaxation is tight , the optimal matching and the optimal dual variables and satisfy the following properties :  1 .  if then 2 .if then 3 .if no edge in is incident on node , then 4 . for all _ factor graph _ of a probability distribution represents the conditional independencies of the distribution .the max - product ( mp ) algorithm is a simple , local , iterative message passing algorithm that can be used ( in an attempt ) to find the mode / map estimate of a probability distribution .nodes and factors pass messages to each other , and nodes maintain `` beliefs '' , which represent the max - marginals . when max - product is applied to problems involving general `` loopy '' graphs , one of the following three scenarios may result :  1.  the algorithm may not converge .2 .  the algorithm may converge , but to an incorrect answer .3 .  the algorithm may converge to the correct answer .  as has been mentioned , herehas been siginifcant work attempting to understand the properties of mp for loopy graphs . for the results in this paper , we will use the following two insights :  1 .  at any time, the belief of the max - product algorithm for a given variable corresponds to the belief at the root of the corresponding _ computation tree _distribution associated with that variable at that time .we describe what this computation tree distribution corresponds to for the weighted matching problem in the next section .2 .  if max - product _ does _ converge , the resulting beliefs are optimal in a large `` local '' neighborhood : let be the assignment as given by the converged max - product and be any other assignment . if the variables assigned different values in and form an induced graph containing at most one cycle in each component , then .the problem of finding can be formulated as the problem of finding the mode of a suitably ( artifically ) constructed probability distribution . in fact , there are in general _ several _ ways to construct this distribution for the _ same _ instance of a graph .we now present one construction .  associate a binary variable with each edge , and let here represents the neighborhood of node in , and is a normalizing constant .the variable can be interpreted as follows : indicates that , while indicates .the term enforces the cosntraint that of the edges incident to node , at most one can be assigned the value `` 1 '' .thus , it is easy to see that if and only if the edges with constitute a matching in . furthermore , the mode of corresponds to the max - weight matching .the factor graph max - product involves messages between variables and factors . in our casethe variables are the edges , and the factors are nodes . thus at any time there will be messages from node ( factor ) to edge ( variable ) , as well as messages .each message will be a length - two vector of real numbers , indexed by 0 and 1 .the message update rules can be simplified to the following : & = & e^{w_{ij } } m_{j\rightarrow  ( i , j)}^t[1 ] \\m_{(i , j)\rightarrow i}^{t+1}[0 ] & = & m_{j\rightarrow ( i , j)}^t[0 ] \\m_{i\rightarrow ( i , j)}^{t+1}[1 ] & = & \prod_{k\in { \mathcal{n}}(i)-j }  m_{(k , i)\rightarrow i}^t [ 0 ] \\m_{i\rightarrow ( i , j)}^{t+1}[0 ] & = & \max { \huge \ { } \prod_{k\in  { \mathcal{n}}(i)-j } m_{(k , i)\rightarrow i}^t [ 0 ] ~~ ,  \\  & & \quad \quad \quad \max_{k\in { \mathcal{n}}(i)-j } m_{(k , i)\rightarrow  i}^t[1 ] { \huge \ } } \end{aligned}\ ] ] also , at every time each edge ( variable ) maintains a belief vector as follows : & = & m_{i\rightarrow ( i , j)}^t[0 ] ~ \times ~ m_{j\rightarrow ( i , j)}^t[0 ] \\b_{(i , j)}^t[1 ] & = & e^{w_{ij } } m_{i\rightarrow ( i , j)}^t[1 ] ~ \times ~ m_{j\rightarrow ( i , j)}^t[1 ] \end{aligned}\ ] ] the defined above can be used to find as follows : first run max - product . at any time and for each edge there will be two beliefs  ] .if max - product converges , assign to each variable the value ( i.e. `` 0 '' or `` 1 '' ) that corresponds to the stronger belief .then , declare the set of all edges set to `` 1 '' to be the max - product output .our proofs rely on the computation tree interpretation of the max - product beliefs .we now describe this interpretation when max - product is applied to as given in ( [ eq : p ] ) .  for an edge  be the _full depth- computation tree rooted at . this is generated recursively : take and to each leaf add as children a copy of each of the neighbors of in , except for the unique neighbor of which is already present in .also , each new edge has the same weight as its copy in the original .the recursion is started with the single - edge tree , both of whose endpoints are leaves .this initial edge is the _ root _ of .consider now the `` full synchronous '' max - product , where at each time every message in the network is updated . in this casethe computation tree for edge at time will be .alternatively , max - product may be executed asynchronously with only a subset of the messages updated in every time slot . in this case will be a sub - tree of . in either case , the computation tree interpretation states at time we have >b^k_e[0] ] if and only if , and <b_e[0] ] for all and < b^t_e[0] ] .then , there exists a matching in such that _ ( a ) _ the root , and _ ( b ) _ has the largest weight among matchings on .let be the set of all edges in that are copies of edges in .by lemma [ lem : matching_corresp ] , is a maximal matching on .also , the root by assumption .the symmetric difference consists of disjoint alternating paths in : each path will have every alternate edge in and all other edges in .let be the path that contains the root .we now show that .recall that the optimal dual solution assigns to each node in a `` dual value '' .associate now with each node in the dual value of its copy in .then , by lemma [ lem : lp_tight ] we have that for each .suppose now that neither endpoint of is a leaf of .in this case , we have on the other hand , we know that ( [ eq : gap ] ) holds for each edge in . adding these up gives by assumption ,the root , so and hence when no endpoints of are leaves .suppose now that exactly one of the endpoints of is a leaf of . in this case, we have that where the last inequality follows from part 4 of lemma [ lem : lp_tight ] also , is assumed to be full up to depth , so this implies that .this means that now , since , this implies that .the final case , where both endpoints of are leaves , works out in the same way , except that now and .thus , in any case , we have that .consider now the set of edges .this set forms a matching on , and has higher weight than .this contradicts the choice of , and so establishes that < b^t_e[0] ] for all .this completes the proof .   in this section we prove that if the lp relaxation is loose then max - product does not converge to the correct answer . before we do so however , we note that this implies a stronger result : that when lp is loose then in fact max - product does not converge at all .[ lem : local_global ] consider the distribution as given in ( [ eq : p ] ) .if max - product converges , then its output exactly corresponds to the true optimal matching .the proof of this lemma uses the `` local optimality '' result of weiss and freeman .in particular , for it turns out that local optimality implies global optimality .this means that it is not possible for max - product to converge to an incorrect answer : it will either not converge at all , or converge to .we do not use this explicitly in the proofs below , but it strengthens the results as mentioned above .  we now proceed with showing that max - product does not converge to the correct when lp is loose . as a first step ,we need a combinatorial characterization of when the lp relaxation is loose .we now make some definitions .we say that a node is _ saturated _ by a matching if there exists an edge that is incident to .a _ blossom _ with respect to a matching is an odd cycle with edges in .note that a blossom has a unique _ base _ : a node not saturated by any edge in .a _ stemmed blossom _ ( w.r.t ) is a blossom , along with an alternating path ( stem ) that starts at the base of , and starts with an edge in .also , should be such that the set remains a matching in .a _ bad stemmed blossom _ is one in which the edge weights satisfy note that it may well be the case that , in which case is just an odd cycle .the following is an example of a bad stemmed blossom .the bold edges are the ones in , the numbers denote the weights of the corresponding edges , and the last node has no edge of incident on it .the blossom in this case is the cycle , and node is its base .the path / stem is .a _ blossom pair _ is two blossoms and and an alternating path between the bases of the two blossoms such that begins and ends with edges in .bad blossom pair _is one in which the edge weights satisfy the following is an example of a bad blossom pair .the following proposition provides a combinatorial characterization of when the lp relaxation is loose , and is crucial to the proof of the subsequent theorem .[ prop : bad_subgr ] if the lp relaxation is loose , then there exists a bad stemmed blossom , or a bad blossom pair , with respect to the optimal matching .  _proof : _ in appendix .we use the presence of these `` bad '' subgraphs in to show that max - product does not converge to the correct answer . before we do so , we need one additional lemma .this states that if max - product converges by step to some matching on , then the optimal matching on the computation tree looks like in the neighborhood of the root .[ lem : convg_tree ] suppose max - product converges to a matching in by step .consider any edge , some and a corresponding computation tree which is full up to depth .let be the max - weight matching on the tree .then , for any edge that is within distance of the root , if and only if its copy in is such that .note that the above lemma also applies to the root of the tree .we are now ready to state and prove the main result of this section .recall that the belief on an edge at convergence is incorrect if either but >b_e[1] ] .  consider a weighted graph for which the lp relaxation is loose .then , the max - product beliefs do not converge to the correct : for any given , there exists a and computation trees such that each is full upto depth , but the beliefs on some of the edges are incorrect .lemma [ lem : local_global ] further implies that in fact in this case max - product does not converge at all ._ proof : _let be the max - weight matching on . since the lp relaxation is loose , by prop .[ prop : bad_subgr ] , there exists either a bad stemmed blossom or a bad blossom pair w.r.t .suppose first that it contains a bad stemmed blossom , and consider some that is in the `` blossom '' part of ( as opposed to the stem ) and also in .from the two nodes of , make maximal alternating paths and that remain in and start out in opposite directions on .for the stemmed blossom example above , if is the edge then the two paths will be and .let , and similarly for . represents the change in the weight of the matching if each edge in were `` switched '' , i.e. their membership in the matching was reversed from its original value .it is easy to see that by assumption is a bad blossom and hence we have that .suppose max - product converges to by step .consider now the computation tree which is full upto depth , where is the number of nodes in .let be the max - weight matching on .lemma [ lem : convg_tree ] implies that will be a projection of in a distance- neighborhood of the root .also , starting from the root , each of and will have a unique copy , say and respectively , in , with . since and are alternating w.r.t . , it follows that and will be alternating with respect to .also , the set forms an alternating path on with respect to , and this begins and ends in nodes unsaturated by .thus , can be augmented by this path : the set will be a matching on .also , the weight gain from doing this augmentation will be exactly , which we know is strictly positive .thus , this shows that is not the optimal matching on , which contradicts the choice of .this means that our assumption about max - product convergence to is incorrect .thus , we see that if there exists a bad stemmed blossom w.r.t . in then max - product does not converge to .a similar argument holds for the case of a bad blossom pair , except that instead of paths and above we now have to look at alternating walks and that live in and are long enough .these walks can then be mapped to an augmenting path on which strictly improves , leading to a contradiction as was seen in the case of the paths and .this completes the proof .the results of this paper can be generalized to the case of perfect matchings , -matchings and perfect -matchings in general graphs , where similar results hold . in this paper max - productis shown to be as powerful as lp relaxation , but it would be more interesting to outline a direct _ operational _ link between max - product and a linear programming algorithm . as an example , shows that for bipartite matching max - product has an operational correspondance with the auction algorithm .also , the form of the message update equations suggests that it can be implemented via an equivalent message passing update rule between just the nodes of the graph , instead of having messages go from nodes to edges and vice versa .more generally , it would be interesting to see if the ideas presented in this paper could be used / genealized to show connections between linear programming and belief propagation in other applications .the author would like to acknowledge dmitry malioutov , whose experiments suggested a strong link between lp relaxation and max - product performance for non - bipartite graphs .dmitry is also responsible for pointing the author to the local optimality result .we now show that if the lp relaxation is loose then there exists in the graph either a bad stemmed blossom or a bad blossom pair , with respect to the optimal matching .let be the optimal ( fractional ) solution to the lp relaxation .let be the set of all edges such that either _ ( a ) _ , or _ and .then , will contain at least one edge , because if all had then the lp would be tight .let be the subgraph of having only the edges in .cycle augmentation _ is any even cycle in which every alternate edge is in .path augmentation _ is any path in which every alternate edge is in , and which begins and ends in nodes unsaturated by . for any augmentation , we have that is also a matching in .thus , if is the unique max - weight matching it has to be that ._ proof : _ let be an augmentation in . by assumption , for all , which implies that for all .thus , there exists some such that decreasing each by and increasing each by represents a valid new feasible point for the lp .the weight of this new point exceeds the weight of by .however this contradicts the optimality of , and thus can not contain any augmentation .let be the longest alternating sequence of edges in , and let and be its endpoints . by the lemma above , both can not be unsaturated .we say that or is a _ saturated leaf _ if it is saturated by and there exist no edges in incident on it .also , note that an endpoint is saturated if and only if its corresponding edge in is also in .   1 .  both and are both saturated leaves + in this case , the constraints at and are loose .so , there exists an such that if all are decreased by and all are increased by then the new solution remains feasible .this new solution will have strictly higher weight than , which is a contradiction .thus this case can not occur . is a saturated leaf and is unsaturated .+ an -perturbation argument like the one above can be used to show that this case too can not occur . is saturated by .but is not a leaf . is either unsaturated , or a saturated leaf .+ since can not be extended , it has to be that all edges in incident to have other endpoints in .let be one such edge . then , forms a stemmed blossom : the resulting cycle has to be odd , and the remaining part of will be a stem whose endpoint is .note that in this case it has to be that the constraint at is loose .4 .  both and saturated by , but are not leaves .+ applying the above blossom argument to both and yields the existence of a blossom pair .thus if the lp relaxation is loose then there exists a stemmed blossom or a blossom pair .now all that is remaining to show is that they are `` bad '' .let be a stemmed blossom in , consisting of blossom and stem .then , there exists some such that if is increased by , is decreased by , is increased by , and is decreased by , then the new solution remains feasible for the lp . also , the new solution weighs \ ] ] more than . for to be the unique optimal of the lp , this has to be strictly negative and thus any stemmed blossom is bad .a similar argument shows that any blossom pair is bad .this finishes the proof of the proposition .        p. rusmevichientong and b. v. roy , `` an analysis of belief propagation on the turbo decoding graph with gaussian densities , '' _ ieee transactions on information theory _ , vol .47 , no . 2 ,pp . 745765 , 2001 .y. weiss and w. freeman , `` on the optimality of solutions of the max - product belief - propagation algorithm in arbitrary graphs , '' _ ieee transactions on information theory _ , vol .47 , no . 2 ,736744 , feb .2001 .m. wainwright , t. jaakkola , and a. willsky , `` map estimation via agreement on ( hyper)trees : message - passing and linear - programming approaches , '' _ ieee transactions on information theory _ , vol .51 , no . 11 , pp . 36973717 , nov .j. feldman , d. karger , and m. wainwright , `` linear programming - based decoding of turbo - like codes and its relation to iterative approaches . '' in _allerton conference on communication , control , and computing _ , 2002 .

Abstract: max - product belief propagation is a local , iterative algorithm to find the mode / map estimate of a probability distribution . while it has been successfully employed in a wide variety of applications , there are relatively few theoretical guarantees of convergence and correctness for general loopy graphs that may have many short cycles . of these , even fewer provide exact `` necessary and sufficient '' characterizations .  in this paper we investigate the problem of using max - product to find the maximum weight matching in an arbitrary graph with edge weights . this is done by first constructing a probability distribution whose mode corresponds to the optimal matching , and then running max - product . weighted matching can also be posed as an integer program , for which there is an lp relaxation . this relaxation is not always tight . in this paper we show that  1 .  if the lp relaxation is tight , then max - product always converges , and that too to the correct answer . 2 .  if the lp relaxation is loose , then max - product does not converge .   this provides an exact , data - dependent characterization of max - product performance , and a precise connection to lp relaxation , which is a well - studied optimization technique . also , since lp relaxation is known to be tight for bipartite graphs , our results generalize other recent results on using max - product to find weighted matchings in bipartite graphs .


numerous statistic studies have been done to uncover the dynamical universal laws in complex system in physical , biological and social areas , such as the reaction dynamics within cells , financial market fluctuations , income distribution , biological species , word frequency , scientific publication , city size , etc.[1 - 19 ] .zipf s law and heaps law are two typical representatives , the coexistence of which have been observed in various systems [ 20 , 21 ] .zipf finds a certain scaling law in the rank of the word frequencies .heaps law reveals another aspect of scaling in natural language processing , according to which the vocabulary size grows in a sub - linear function with document size .it is also found that zipf s law and heaps law holds for different languages with different characterization exponents which means more complicated statistical features [ 22 - 27 ] .moreover , similar results have been recently reported for the corpus of web texts , including collaborative tagging , social annotation , internet search result and computer programs[28 - 32 ] , which indicates universality of zipf s law and heaps law beyond natural language systems .  in this paper , we focus on the word frequency in spoken language . using the speech transcriptions from the speech british national corpus on daily conversations of various subjects and ten classic books for comparison , we show ( i ) the word frequency distribution vs. ranking obeys the zipf s law and growing of distinct words obeys heaps law ; ( ii ) the coefficients of zipf s and heaps laws have significant differences between written texts and speech transcriptions ; and ( iii ) on the basis of the above study, a generalized model is proposed to simulate the growing dynamics and construction mechanism of spoken and written languages and the difference between speech and written texts .empirical observations and model simulations agree well with each other .* experiments . * two classes of data sets were analyzed in this article : ( i ) ten speech transcriptions of the speech british national corpus [ 33 ] from phonetics laboratory , and ( ii ) ten classic books written by different authors in different era .the transcriptions of daily conversation from bnc database are selected to have comparable length as the books .as shown in table 1 , the total lengths of the text are 27341 to 268843 for books , and 25281 to 124802 for speeches .there are relatively less distinct words being used in speeches , ranging from 2158 to 5815 in the selected data sets , and from 2572 to 29020 in books ( see details about data in methods ) .firstly , we analyzed the probability distribution of word frequency .we analyzed each book and speech transcription .the probability distribution of word frequency can be described as a power - law between the word frequency and its probability density ,    where is the power - law scaling exponent .1 shows the probability distribution of word frequency of two sets of data , the result of book is in fig .1(a ) and speech is in fig .1(b ) , which both follow power - law .the goodness of fit is 0.9952 and 0.9944 respectively , and fitting region is ( see details of the goodness of fit in methods ) .all the validation results are listed in table s1 .we found significant difference of scaling exponents between books and speeches ( book : , speech : , ) .   vs. word frequency in log - log scale for ( a ) books and ( b ) speech transcriptions .the fitting regions are .,title="fig : " ]  vs. word frequency in log - log scale for ( a ) books and ( b ) speech transcriptions .the fitting regions are .,title="fig : " ]  zipf found a power - law relation between the work frequency , and its corresponding rank , as    fig .2(a ) presents the plot of word frequency distribution for one book and one speech transcription .the word frequency distributions can be divided into two parts , in the first part , which is corresponding to the high frequency words , the relative word frequency of books is less than that of speech . in the second part ,which is corresponding to the low frequency words , the of books is larger than that of speech .the decay of the second part for speech and book both follow power - law with the goodness of fit 0.9985 and 0.9992 respectively and we found significant difference in the decay exponents between books and speeches ( book : , speech : , , see table s1 ) .   of one book and one speech transcription , with a fitting region .( b ) comparison of the growth of the number of distinct words versus the text length between one book and one speech transcription , fitting region .,title="fig : " ]  of one book and one speech transcription , with a fitting region .( b ) comparison of the growth of the number of distinct words versus the text length between one book and one speech transcription , fitting region .,title="fig : " ]  fig .2(b ) reports the growth of the number of distinct words n(t ) versus the text length t. both books and speeches follow heaps law    and can also be divided into two parts . in the first part , of the speeches is very close to that of the books , corresponding to the linear increasing region . in the second part , the sub - linear increasing region, we can see that of the books is bigger than those of the speeches , which indicates that the growth speed of new words in speech is lower than in book .we calculated the slope of the second part of curve of ten books and ten speech transcriptions as a linear approximate fitting in log - log scale , and the slope values of each subject were listed in table 1 .the -test shows there is a significant difference between the slope of books and speeches ( book : , speech : , ) , which both follow power - law with the goodness of fit is 0.9988 and 0.9980 respectively , and fitting region is ..the basic statistics of the ten speech transcriptions and the ten books . is the total length of the text measured in number of words , and is the total number of distinct words .the slope of of the ten speech transcriptions and the ten books ( local linear approximate fitting in log - log scale , fitting region ) [ cols="^,^,^,^,^,^,^,^ " , ]   fig .3 are the summary of mean and standard deviation of the scaling exponents , and of books and speeches .    * model .* we test whether the rich - get - richer mechanism , also named preference attachment mechanism [ 34 - 37 ] works for spoken language generating process .we denote the average probability that a character appeared times will appear again ( see methods how to measure ) . as shown in fig .4 , for all the books and speeches increase proportionally with , indicating a rich - get - richer effect like the preferential attachment in evolving scale - free networks .   versus in a log - log scale for the representative ( a ) book and ( b ) speech , and the goodness of fit are 0.9477 and 0.9575 respectively.,title="fig : " ]  versus in a log - log scale for the representative ( a ) book and ( b ) speech , and the goodness of fit are 0.9477 and 0.9575 respectively.,title="fig : " ]  we propose a generalized model to further simulate and investigate the empirical observations as an extensive yule - simon model [ 38 - 43 ] .the process of language construction can be modeled as follows . at every time step, a word will be appended to the text , either by generating a new word , or selecting one from the text that already generated .we propose that the growing dynamics revealed by heaps law can be defined as the probability of new word generation , which can gradually change with the text length .the fomula of the probability can be determined according to the specific application . in this investigation of language construction , we set it as :    while with probability , one word is copied from the text that already generated , where word will be chosen is determined by rich - get - richer mechanism .let be the number of appearance of word at time step , then at next step , the word will be selected with the probability    parameter provides a parameter to modulate the strength of preferential attachment . in this proposed model ,the probability of new word generation depends on text length , and decreases monotonously with text increasing , inspired by the empirical observation of . andthe word will be reused according to the rich - by - richer rules , which gives the zipf s law .5 reports the simulation results for one book and one speech transcription using proposed model .all three scaling properties can be very well captured by the model .for all the ten books and ten speech transcriptions , the parameters are as follows : , book : , speech : , ; , book : , speech : , ; , book : , speech : , ( see table s2 and fig .s1 ) .  , and ; and for speech kbg , and .,title="fig : " ] , and ; and for speech kbg , and .,title="fig : " ] , and ; and for speech kbg , and .,title="fig : " ]  , and ; and for speech kbg , and .,title="fig : " ] , and ; and for speech kbg , and .,title="fig : " ] , and ; and for speech kbg , and .,title="fig : " ]previous statistical analyses about human languages mostly concentrated on written texts where language consists of a huge number of words .in contrast , speech languages consisting of fewer words received less attention .the empirical results of the comparison between books and conversations indicate that i ) the speech transcriptions also obey zip s law and heaps law , but with different exponents compared with books .ii ) when the content length of speech grows , the emergence of new words does not increase as much as in books , the heaps law is also deviated from linear behavior .iii ) in speech , the usage of words are much more concentrated on some words , which leads to the larger probability of high frequency rank words than in books .we can further explain the possible reasons .i ) book authors could create new words according to his own written style .new words may result from new techniques , new biological species , or new names .however , generally we seldom give birth to a new word during speech .ii ) book novel needs personalized language to express the characteristics of different persons in the story .using complex grammatical long sentences to express the thoughts .changing the style of words expression with different scenarios .take hamlet as an example , claudius s speech is rich with rhetorical figures , as is hamlet s and , at times , opheliaswhile the language of horatio , the guards , and the grave diggers is simpler . while a great speech puts the occasion , the audience , and the speaker together in an unforgettable way .most estimated the number of words per minute around .so the speaker always choose a more simple way to use words .the currently reported regularities from the well - known zipf s and heaps laws point of view , can be reproduced by considering decrease of new words emergence with the length of generated text in a nonlinear process , which can be formulated according to the specific growing dynamics under study .in addition , the differences we observed in word frequency distribution and zipf s law between speech and books indicate different strength of preferential attachment , which is also considered in the presented model .simulation results confirm that the scaling properties of the complex dynamics of language construction and organization can be very well captured by the proposed model , and the differences between spoken language and written language can also be accounted for by different parameter settings .we further hypothesize that the proposed model of construction and organization in complex system with preferential attachment mechanism can also be applied to other complex system in physical , biological and social areas .* data description * 10 books are analyzed in this article : no.1 , alice s adventures in wonderland , written by lewis carroll ; no.2 , the adventures of tom sawyer , written by mark twain ; no.3 , a christmas carol , written by charles dickens ; no.4 , david crockett , written by john s. c. abbott ; no.5 , an enquiry concerning human understanding , written by david hume ; no.6 , hamlet , written by william shakespeare ; no.7 , the hound of the baskervilles , written by sir arthur conan doyle ; no.8 , moby - dick : or , the whale , written by herman melville ; no.9 , the origin of species by means of natural selection , written by charles darwin ; no.10 , ulysses , written by james joyce .these books cover disparate topics and types and were accomplished in far different dates .the basic statistics of these books are presented in table 1 .this corpus of all the ten chinese books consisting of about 1,002,504 total words and 100,335 distinct words .the transcriptions of the speech british national corpus from phonetics laboratory are used in this article : the british national corpus ( bnc ) is a 100 million word collection of samples of written and speech language from a wide range of sources , designed to represent a wide cross - section of british english from the later part of the 20th century , both speech and written .the speech part consists of orthographic transcriptions of unscripted informal conversations ( recorded by volunteers selected from different age , region and social classes in a demographically balanced way ) and speech language collected in different contexts , ranging from formal business or government meetings to radio shows and phone - ins .we selected 10 transcriptions of daily conversation from bnc database which have comparable length as the books .this corpus of all the ten speech text consisting of about 747,454 total words and 43,468 distinct words .* goodness of linear fit statistics in log - log scale * the original data , and are calculated in linear intervals , in log - log scale a lot of data cluster at large scales ( see fig .s1 for the data clustering at large scales ) .if the fitting process is performed on the original data , the data points at large scales will dominant the cost function and the measure of goodness of fit , causing bias towards large scales when fitting data in log - log scale .thus , before the fitting process , we resample the original data to be equally distributed in log scale , as    where is the number of the original data , and b can be any real number that greater than 1 .then , we fit the resampled data in a least - squares sense in log - log scale , as to minimize the following cost function   ^ 2 , \label{equ : fit2}\ ] ]  and the statistic measures of the goodness of fit in log - log scale is defined following the definition of r - square as   ^ 2 } { \sum_{i=1}^{n } [ \log(y'(i))-\mathrm{e}(\log(y'(i)))]^2 } , \label{equ : fit3}\ ] ]  where e( ) denotes the calculation of mean value , and n is the number of the resampled data .* preferential attachment . * for each speech or book text , we divide it into two parts : part i contains a fraction of words appeared and part ii contains the remain fraction of words . for word in partii , if appeared times in part i , we add one to whose initial value is zero .accordingly , is the number of words in part ii that appeared times in part i. dividing by the number of distinct words that appeared times in part i. in this paper , we show the results for .we thank the support from the fundamental research funds for the central universities of chinananjing university scholar exchange project , natural science foundation of china ( 61271082 , 61201029 ) , priority academic program development of jiangsu higher education institutions ( papd ) , and natural science foundation of jiangsu province ( bk2011759 ) .c.h.b conceived the research .r.k.l and q.l.m analysed the data , c.h.b and q.l.m designed and simulated the model .all authors wrote and revised the manuscript .the authors declare no competing financial interests .searls , d. b. the language of genes . _nature _ * 420 * , 211217 ( 2002 ) .gao , j. , hu , j. , mao , x. & perc , m. culturomics meets random fractal theory : insights into long - range correlations of social and natural phenomena over the past two centuries ._ j. r. soc .interface _ * 9 * , 19561964 ( 2012 ) .                \34 .clauset , a. & moore , c. accuracy and scaling phenomena in internet mapping .lett . _ * 94 * , 018701 ( 2005 ) .35 . jeong , h. , neda , z. & barabasi , a .-measuring preferential attachment for evolving networks ._ europhys .* 61 * , 567572 ( 2003 ) .

Abstract: human language , as a typical complex system , its organization and evolution is an attractive topic for both physical and cultural researchers . in this paper , we present the first exhaustive analysis of the text organization of human speech . two important results are that : ( i ) the construction and organization of spoken language can be characterized as zipf s law and heaps law , as observed in written texts ; ( ii ) word frequency vs. rank distribution and the growth of distinct words with the increase of text length shows significant differences between book and speech . in speech word frequency distribution are more concentrated on higher frequency words , and the emergence of new words decreases much rapidly when the content length grows . based on these observations , a new generalized model is proposed to explain these complex dynamical behaviors and the differences between speech and book .   * keywords : * speech transcription , zipf s law , heaps law , preferential attachment , complex system