consider a large - scale publish - subscribe application that is characterized by a large number of information flows , as well as a large number of subscribers .each information flow generates messages which must be delivered to an interested subset of subscribers .subscribers are interested in different , yet possibly overlapping , subsets of the information flows .naturally , an individual information flow may be required by many subscribers .a typical example is a financial market data dissemination system , where the flows can be stock quotes ( of which there are tens of thousands ) , commodity prices , etc . , and subscribers are traders , analysts and so on ( in hundreds ) .each subscriber is interested in a different portfolio .one of the two common approaches in the above dissemination scenario is to utilize ip multicast to transmit the data . in this workwe assume that ip multicast service is supported in the enterprise network .given that overlaps between subscribers interests are not rare , transmission costs can be reduced by grouping information flows into groups , and using multicast to disseminate these flows to subscribers .this mechanism requires two mappings : one between flows and groups ( mapping of a flow to one or more multicast group ) , and another mapping between users and multicast groups ( such that each subscriber gets all the information she is interested in ) .the problem of finding the mappings which minimize consumption of network resources is termed _`` the channelization problem '' _ .  using multicastas a mean of dissemination has some limitations . typically , there is a limited amount of multicast addresses which can be used , due to the memory and computational overhead of the network devices .  in our setting ,the number of flows is much larger than the number of available multicast groups , which means that a one - to - one mapping of flows to multicast groups is not possible .thus , different flows have to be batched into the same multicast group . asa result subscribers may receive data they are not interested in and which they must filter . as shown in  ,filtering in the end hosts is one of the reasons for reduced performance in a high bandwidth enterprise network .a second common approach is to use point - to - point connections , where each publisher transmits the information required using unicast .the main drawback of solely using unicast is the poor utilization of network resources when many subscribers are interested in the same data flow . in this case, the transmitter has to transmit the same data many times to different users , which results in a waste of transmitter resources like bandwidth , cpu and memory as well as wasted network bandwidth .  in the current paper, we propose a novel hybrid approach , which combines both unicast and multicast transports . in our approach, we allow a flexible allocation of flows into unicast and multicast connections , avoiding the inherent drawbacks of using a single scheme .thus , we gain high efficiency when many subscribers are interested in the same data flow by utilizing multicast , and use unicast to reduce unneeded filtering , whenever the subscription to certain flows is relatively rare .we define the hybrid unicast - multicast problem as an optimization problem , and explore several heuristics to solve it . using extensive simulations , we compare different approaches for solving the hybrid problem and identify which perform best , under different scenarios . as an additional contribution, we explore different algorithms for solving the related channelization problem , which is np - hard , and identify a single algorithm which outperforms the others .the paper is organized as follows .section [ sec : rel_work ] overviews the related work and explains the novelty in our hybrid approach .section [ sec : model ] describes the problem model and formally defines the hybrid problem , showing it is a np - hard problem .section [ sec : algos ] presents our proposed heuristics for solving the hybrid problem .section [ sec : exp_res ] gives extensive experimental results which compare the different heuristics and their operation under various real - world scenarios .we conclude in section [ sec : conc ] .publish - subscribe systems have been the target of extensive research in the last years .research has focused on the problem of disseminating data efficiently to interested users .two main paradigms were proposed : content - based multicast and subject - based multicast  ) .different extensions to the paradigms include  where a hybrid approach for content - based and subject based dissemination is proposed .another example is  which proposes a solution for a setting in which dynamic changes of the multicast groups is required . in  content - based disseminationis implemented using ip multicast .one of the main challenges when considering subject - based multicast is in solving the channelization problem ( ) .previous approaches map flows into multicast groups , while the current paper allows for both multicast and unicast transmissions . in section[ sec : exp_res ] we empirically compare several algorithms for solving the channelization problem , identifying a single algorithm which outperforms the others .  a closely related work to ours is dr .multicast which proposes to use unicast as well as multicast in a data center information dissemination scenario . however , focuses on the management and stability issues of ip multicast in the data - center , and does not explicitly explore the question of mapping flows into multicast and unicast in a quantitative manner . to the best of our knowledge ,we are the first work which formally defines the problem as an optimization problem , and explores several heuristics to solve it .we use the following notations as in  .* let be the number of users .* let be the number of flows .* let be the number of mulitcast groups .* let denote the interest matrix : * let denote the mapping from flows to multicast groups : * let denote the mapping from multicast groups to users : * let denote the unicast matrix : * let denote the rate of the flows where is the rate of flow .   given users , flows , multicast groups , a vector of flow rates and an interest matrix , the channelization problem  aims at finding two mapping matrices that minimize the cost of transmission ( using only multicast groups ) , under the constraint that each user receives all the flows it is interested in .a schematic diagram of the channelization mappings is given in figure [ fig : channelization ] .+  to formally define the cost function , let be real positive numbers ,   the cost consists of two terms ; the first sums all transmission received by subscribers . for each user it sums the number of times receives any given flow , times the rate of flow .the second term sums the transmission of the senders ; that is , each flow is summed according to the number of multicast groups it is transmitted to , times the rate . are factors which weight the two terms .the channelization problem is defined as :  in other words , given a set of users , a set of multicast groups , a set of flows , an interest matrix and a flow - rate vector ; find that minimize under the constraint that .below we model our hybrid framework as an optimization problem . unlike the original channelization problem , the transmitters may send flows using unicast .that is , any flow can be transmitted using unicast to any user . in the hybrid problemthe cost function is composed of three terms :   the additional term represents the cost of all the flow -user pairs such that flow is sent using unicast to user , multiplied by the cost of transmission .the cost of transmission of a flow consists of the cost of sending the flow ( ) , the cost of receiving the flow ( ) , and the cost incurred by the unicast mechanism , ( additional memory requirements , etc ) . for the rest of the paper , we assume that the transmitting and receiving costs are equal ( ) and that the unicast cost equals their sum ( _ i.e. _ , ) .  using the cost function the hybrid channelization problem can be formally defined .given users , multicast groups , flows , an interest matrix and a flow - rate vector , the hybrid channelization problem is defined as :   the constraint requires that each user requesting flow will either receive via unicast or via a multicast group listens to .( may receive flow via both multicast and unicast ; however , that would be wasteful . )the hybrid channelization problem is np - hard .  in was shown that the non - unicast problem is np - hard , therefore the unicast channelization problem can be reduced to the non - unicast channelization problem as a proof of its hardness .the reduction is simple : given as input to the channelization problem , construct an input to the hybrid channelization problem which is exactly the same , with a single modification . setting to be large than ensures that any solution does not have a lower cost than .thus , the minimal cost is the same as in the non - unicast setting .we propose the following two - step framework for solving the hybrid problem . in the first step ,we solve the channelization problem , without assigning any unicast flows . in the second step , we heuristicallyselect some of the flows to be sent using unicast , and update the subscription matrix accordingly .this process can be repeated several times , as long as the system cost is reduced .simulation results show that repeating the process does not significantly improve system cost , while having a high computational cost .the above process can be viewed as starting from a solution that uses only multicast , and then using unicast to greedily improve the solution .alternatively , one can start with a solution that uses only unicast ( _ i.e. _ , ) , and greedily improve it by moving flows to multicast .our simulations show that the first framework operates better than the latter one ; while both of them improve upon previous non - hybrid solutions .previous work  discuss several heuristics for solving the channelization problem .et al._examine several heuristics , among them , random assignment and user and flow based merges .et al._proposed a variant of the k - means algorithm which greedily minimizes the cost on each iteration .  in this work, we extensively compare the different previous approaches , while exploring new algorithms .we have utilized an algorithm from the data mining domain , called binary matrix decomposition ( bmd ) which is originally used for two - sided binary clustering of documents and keywords into document classes .the basic idea is that the global cost function for minimization is :  which means we are looking for a decomposition of the interest matrix into two binary matrices so that the euclidian norm between and is minimized .an alternating algorithm is derived by starting with an initial guess , solving which minimizes the cost function and then continuing in rounds .there are some drawbacks in using this algorithm : first , it does not support variable flow rates .second , it allows for some flows to be missing . despite those drawbacks it has reasonable performance when operating on small systems ( _ i.e. _ , 200 flows , 10 multicast groups , 100 users )however , when operating on larger systems ( _ i.e. _ , 10000 flows , 100 multicast groups , 250 users ) it does not improve upon a random selection of a solution. therefore , we have omitted the experimental results of the bmd algorithm from the following graphs .we have also utilized the matlab k - means algorithm which is a two phase algorithm . in the first phase pointsare reassigned to their nearest cluster centroid , all at once , followed by recalculation of cluster centroids .the second phase uses `` on - line '' updates , where points are individually reassigned while reducing the total cost function , and cluster centroids are recomputed after each reassignment .we further investigated an interior point algorithm .starting from the original problem formulation by adler __ , the binary mapping matrices and are relaxed to be in the range . the constraints that and are incorporated into the cost function using the log - barrier technique ( ) and then the newton method is applied .after convergence , the solution is rounded to receive binary and .unfortunately , the interior point method did not perform well in practice .some of the reasons are that the problem is neither concave nor convex .we have usually received a good fractional solution , but when the solution was rounded to the closest integer solution , it did not compare favorably to the other algorithms .therefore , we have omitted the experimental results of the interior - point algorithm from the following graphs .  in total , we have examined five different algorithms for solving the channelization problem .table [ tab : algos ] summarizes the tested algorithms .regarding their running time , not surprisingly , the random assignment is the lightest algorithm with running time of ( setting each flow to a random multicast group ) plus for going over all users and assigning them to groups such that they will receive all required flows .the two k - means variants as well as the bmd algorithms have a similar running time , where is the number of iterations ( typically five on problem sizes of thousands ) , since for each flow they go over all possible assignments of groups by taking the minimal cost .the interior point method running time is dominated by the hessian inversion in the newton step ..examined algorithms for solving the channelization problem and their running time .[ cols="<,<",options="header " , ]   different ways of choosing flow - user pairs can be used .we concentrated on two different types of heuristics : flow based and user based .flow based heuristic means that each flow is either sent to all users that are interested in it via unicast , or transmitted to all of them via multicast ; one can either remove the `` heaviest '' flow or the `` lightest '' flow ( in the sense of the amount of bandwidth required to transmit that flow to all users interested in it ) .clearly , we expect the lightest - flow approach to outperform the heaviest - flow approach ; this has been validated by our simulations , and in the following graphs we will consider only the lightest - flow approach .user based heuristics means that all flows sent to user are sent via unicast .that is , if user receives any flow using unicast , then any other flow that is sent to is sent using unicast .similar to the case of flow removal , we can choose to remove the `` heaviest '' or `` lightest '' user ( here `` heavy '' and `` lightweight '' means the total bandwidth required to transmit all flows user is interested in ) .our simulations show the heaviest - user approach outperforms the lightest - user approach ; the reason lies in the fact that heavy users listen to many multicast groups , and thus incur large overhead in filtering . in the following graphs we show the heaviest - user approach only .  to sum up, we have tested the heuristics of removing the heaviest / lightest flow / user from , and moving it to .in addition , each of the above options was tested twice : once by finding a single pair then removing as many flows / users from as possible ( termed `` non - iterative '' ) ; and once by finding a pair , removing a single flow / user from , then finding a new pair ( that optimizes the modified ) , removing another flow / user from the altered , and so on ( as long as the cost function was minimized ) ; termed `` iterative '' .our simulations have shown the non - iterative approach operates almost as good as the iterative , with significantly reduced computational effort .thus , the following graphs depict only the non - iterative runs .in addition , we have tested several other heuristics .the basic idea is to remove a flow / user in a greedy way , _i.e. _ , repeatedly move to unicast the user / flow / flow - user pair that minimizes the total cost ( eq . [ hc ] ) , until cost does not decrease or bandwidth for unicast is fully utilized .we call those heuristics greedy user , greedy flow and greedy flow - user pair accordingly .  in practice, the flow - user pair heuristic did not perform well , while having a high computational cost .thus , it is not shown in the graphs . to sum up, we have tested in total eleven different heuristics . in the following section, we present the simulations results for the best - performing among these heuristics .we have experimented with three possible initializations of the user interest matrix .the first one is _ random _, where each user uniformly selects % 3 of the flows .the second one , _ market distribution _, is based on a model of subscription patterns in financial messaging systems  .this model is based on stock market symbol rates collected from the new york stock exchange ( nyse ) .the matrix was composed of 10,000 symbols divided into 10 markets , and 250 users .each user was interested in 4 markets , and chose some of the symbols in each selected market .the flows within a market are distributed exponentially , and the markets are distributed using zipf distribution .the _ market distribution_determines the flow rate as well .  figure [ fig : user - interest - matrix ] shows an example of a user interest matrix ( top ) , and the relative message rate of each symbol ( bottom ) , according to the _ market distribution_.   and message rate  according to the market distribution model.,scaledwidth=40.0% ]  the third initialization to the matrix uses a subscription pattern captured from an ibm s websphere test cluster .in it there are 79 processes subscribed to over 6100 topics .subscription to the topics is entirely automatic , influenced by the configuration and load incurred upon the cell .as can be seen in figure [ fig : user - interest - matrix2 ] , the resulting interest matrix is clearly different from the one generated by the model of human behavior in financial markets ( see figure [ fig : user - interest - matrix ] ) . importantly , many topics have identical audiences , which perfectly lends itself to multicast channelization .     among the algorithms listed in table [ tab :algos ] , only the k - means and the interior - point method take the flow rates into consideration .thus , only the k - means was plotted twice , once with equal rate and once with rate derived by the _ market distribution _ , as shown in figure [ fig : non - unicast ] . using equal rate , both k - means and matlab k- means have a superior performance .however , using _market distribution_rate , the k - means algorithm has a noticeably superior performance over all others .    in all graphsshown , the y axis represents percentage cost from perfect multicast , where the term perfect multicast refers to the cost of transmission using multicast transport only ( with no unicast ) , assuming there are unlimited number of multicast groups .thus , perfect multicast means that each user receives exactly all flows it is interested in , each flow is transmitted only once and there is zero filtering in the network .  in the hybrid setting , we allow some of the traffic to be transmitted using point - to - point connections .we have tested different heuristics for moving traffic from multicast to unicast ( see subsection [ sec : choosesys ] ) .figure [ fig : unicast_methods ] compares the top heuristics : lightest - flow , heaviest - user , greedy flow and greedy user .as can be seen , allowing some of the data to be unicasted reduces the cost .evaluated using the market distribution , it seems that the greedy - user heuristic outperforms the greedy - flow heuristic .however , this result is overturned when evaluating using the websphere distribution ( in the sequel ) .thus , the relative competitiveness of these two heuristics depends of the nature of the interest matrix .the interest matrix represents the flows each user is interested in .the performance of the different heuristics is highly dependent on the content of which represents the characteristics of the instance of the problem . in figure[ fig : interest_matrix ] the lightest - flow heuristic is evaluated with different interest matrices : a random interest matrix where each flow has the same rate , a _ market distribution_interest matrix where all flow have a fixed same rate , and a _ market distribution_interest matrix where the rates are also according to _market distribution_. as can be seen in figure [ fig : interest_matrix ] , the algorithm performs best when running on a _ market distribution_interest matrix ; _ i.e. _ the heuristic is optimized for the expected distribution of a real - world financial market application .this happens because of the underlying zipf probability , where the top flows are requested by a large number of users .this makes the clustering of top flows into multicast groups easier .figure [ fig : scalability ] shows how the different heuristics perform as the size of the system increases .each point in the figure represents a different system : for point , the system consists of flows , users while the number of multicast groups is fixed to 50 .we did not scale the number of multicast groups since it is usually dictated by the networking hardware .the relation between the different heuristics is mostly preserved at different system sizes .an interesting exception is point , in which the greedy flows outperforms the greedy users heuristic .this effect is not surprising as different systems ( specifically , the ratio between flows , users and multicast groups ) can change the relative efficiency of the different heuristics .    to show the behaviorial difference of the heuristicswhen running on a mechanical subscription trace , we have ran the different heuristics on the ibm websphere distribution ( see figure [ fig : unicast_methods2 ] ) . as can be seen , when the subscription patters closely overlap , the flow based heuristic outperform the user - based heuristics .it is interesting to note that the heavy - user heuristic actually increases the cost , since this heuristic moves the heaviest user and does not consider the cost of the move .in addition , the greedy user and greedy flow heuristics reach their maximal improvement at very low unicast bandwidth .this phenomena is due to the structured nature of the interest matrix , incurred by the mechanical subscription pattern .figure [ fig : unicsat0 - 100 ] represents well the benefits of using our hybrid approach .the greedy heuristics is forced to use a given percentage of unicast bandwidth ( the x - axis ) , using the websphere subscription pattern . using the hybrid approach, the greedy flow heuristic improves upon both the multicast only and unicast only schemes . the total cost of transmission is reduced in a way which is not possible using a single scheme .we have experimented with different heuristics for selecting which of the data should be transmitted using unicast . under the stock market model ,the best heuristics are greedy heuristics which repeatedly move a single user / flow from multicast to unicast to minimize the total cost . in thissetting , the distribution leads to several multicast groups which carry a large number of non - identical heavy flows .thus , a user that is interested in any heavy flow might be forced to receive it via a multicast group that carries other heavy flows that he does not need , leading to a high filtering cost . in this case , the gain of moving a single user to unicast outweighs the gain that might be achieved by moving the best flow to unicast , because the best flow to be moved is usually fairly light weight .therefore , the heuristic of greedily moving users from multicast to unicast works well in this setting .  the second scenario we tested consisted of a user interest matrix from a websphere test cluster . as the users of this problem are software / script based , their interests are homogenous .thus , many users can use the same multicast group with no need for filtering .therefore , a flow which is of interest to a few users can incur heavy cost on filtering , if it is assigned to a multicast group that many users listen to .this property causes the user based heuristics to perform poorly , while the flow based heuristics perform well .in other words , the greedy - user and greedy - flow schemes `` thin - out '' the interest matrix by removing rows and columns , respectively , making the resulting interest matrix more amenable to channelization .the relative competitiveness of these heuristics depend on the structure of the interest matrix .this paper analyzes the hybrid channelization problem .we formally define the problem as an optimization problem and propose efficient heuristics for solving it .our general framework starts from a solution to a non - unicast problem and tries to improve it by allowing some of the data to be transmitted via unicast .similarly , we start from a solution which utilizes only unicast , then improve it by allowing some of the data to be transmitted via multicast .  we have tested our heuristics against two different real - world scenarios .first is a simulated brokers interest in financial market data and the second is mechanical subscription pattern captured from an ibm websphere test cluster .five different algorithms for solving the non - unicast channelization problem where examined , and a single algorithm , the k - means algorithm was identified to perform the best in all settings .  in totalwe have experimented with eleven different heuristics .the greedy heuristics ( that improve the cost function directly ) performed better than the others .however , greedy heuristics should be taken with a salt of grain , as different problems incur different distributions on the user interest matrix and on the rate of the flows .thus , different heuristics may perform differently as the problem context changes .  to conclude , by allowing a combination of multicast and unicast transmissions , we gain in reduced host and network resource consumption .it seems that the performance of a publish subscribe system is highly depended on the subscription patterns .our hypothesis is that user based heuristics combined with the flow based heuristics cover a large range of problems .thus , we provide a range of heuristics that can be used to practically deploy a publish subscribe system efficiently .f. cao and j. p. singh .medym : match - early and dynamic multicast for content - based publish - subscribe service networks . in _icdcsw 05 : proceedings of the fourth international workshop on distributed event - based systems ( debs ) ( icdcsw05 ) _ , pages 370376 , washington , dc , usa , 2005 .ieee computer society .isbn 0 - 7695 - 2328 - 5 - 04 .s. ganguly , s. bhatnagar , a. saxena , r. izmailov , and s. banerjee . a fast content - based data distribution infrastructure . in _infocom 2006 .25th ieee international conference on computer communications . proceedings _ , pages 113 , 2006 .   b. n. levine , j. crowcroft , c. diot , j. garcia luna aceves , and j. kurose .consideration of receiver interest for ip multicast delivery . in _proceedings ieee infocom _ , volume 2 , pages 470479 , mar 2000 .t. li . a general model for clustering binary data .kdd 05 : proceedings of the eleventh acm sigkdd international conference on knowledge discovery in data mining _ , pages 188197 , new york , ny , usa , 2005 .isbn 1 - 59593 - 135-x .l. opyrchal , m. astley , j. s. auerbach , g. banavar , r. e. strom , and d. c. sturman .exploiting ip multicast in content - based publish - subscribe systems . in _ifip / acm international conference on distributed computing platforms _ , pages 185207 , 2000 .y. tock , n. naaman , a. harpaz , and g. gershinsky .hierarchical clustering of message flows in a multicast data dissemination system . in s. q. zheng , editor ,_ iasted pdcs _ , pages 320326 .iasted / acta press , 2005 .isbn 0 - 88986 - 525 - 6 .y. vigfusson , h. abu - libdeh , m. balakrishnan , k. birman , and y. tock .multicast : rx for datacenter communication scalability . in _hotnets vii : seventh acm workshop on hot topics in networks _ , calgary , canada , october 2008 .

Abstract: one of the main challenges in building a large scale publish subcribe infrastructure in an enterprise network , is to provide the subscribers with the required information , while minimizing the consumed host and network resources . typically , previous approaches utilize either ip multicast or point - to - point unicast for efficient dissemination of the information . in this work , we propose a novel hybrid framework , which is a combination of both multicast and unicast data dissemination . our hybrid framework allows us to take the advantages of both multicast and unicast , while avoiding their drawbacks . we investigate several algorithms for computing the best mapping of publishers transmissions into multicast and unicast transport .  using extensive simulations , we show that our hybrid framework reduces consumed host and network resources , outperforming traditional solutions . to insure the subscribers interests closely resemble those of real - world settings , our simulations are based on stock market data and on recorded ibm webshpere subscriptions .   performance  ip - multicast , publish - subscribe


some 75 years ago zipf found that the word frequency of a language has a very particular `` power - law like '' distribution .this phenomena is best known as zipf s law and states that the number of occurrences of a word in a long enough written text falls off as where is the occurrence - rank of a word ( the smaller rank , the more occurrences )   .how well is this power law obeyed ?what is its origin ?what does it imply from a linguistic and cognitive point of view , if anything ?simon in ref . emphasized that the fact that `` power law '' distributions occur in a wide range of seemingly unrelated phenomena suggests a general underlying stochastic nature .in particular he devised a general stochastic model for the writing of a text , the _simon model _ .the random element in this model is tied to the actual process of evolving the text and not to a property of the language itself .simon model _ and its stochastic evolution mechanism has since its first appearance turned up in many disguises such as rich - get - richer models and preferential attachment .an alternative view was taken by mandelbrot who proposed that zipf s law of word frequencies could be associated with the collective language itself rather than with the evolution of a particular text .in particular he proposed that the `` power - law like '' distribution could be linked to an optimization of a letter - combination information .however , miller in ref . showed that a power law distribution of words in a collective language does not _per se _ requer any optimization , which gave rise to the metaphor of a monkey randomly writing on a typewriter .all these proposed explanations presumes that the `` power - law like '' distribution says nothing about the syntax , grammar and context correlations of a written text . yetthe word correlations are , of course , essential for the meaning of a text .  in the present paper we focus on the function , the number of distinct words which occur precisely times in a written text .the correspondence of zipf s word rank power law is for this quantity  .we here focus on the properties of single novels , each novel written by a single author . in this waywe ensure that both the evolution aspect of the text and the properties of the language always relates to the very same text . from this perspectivea novel can perhaps be regarded as a fingerprint of the author s brain .we demonstrate that the text of a novel display certain general features and show that these features are shared with a simple null model which we call the random book .in section 2 we describe some general characteristic features which the text of a novel display . for claritywe choose one typical novel as an illustrative example . in appendixa we include data for a collection of novels in order to illustrate the generality of the conclusions . in section 3we discuss the random book transformation which describes how the word - frequency distribution changes with the length of the text analyzed .it is shown that a real novel to good approximation transforms in the same way .it is also shown that the random book transformation can be used in order to obtain a sharper determination of the word - frequency distribution of a novel .section 4 contains our summary and concluding remarks .examples of key characteristics of the word frequencies in a novel are as follows : the most obvious is the word - frequency distribution of the _ complete _ novel .a word is in this context defined as a group of letters separated by blanks .if the book contains  distinct ( different ) words and a total of  words , then is the probability that a word , which you pick randomly in the book , is occurring -times in the book .this means that where  is the maximum number of times a distinct word appears in the book and also that , which is the average number of times a word occurs in the book .the function is the word - frequency distribution and is often very broad and more or less `` power - law like '' , with , over a substantial region .this is illustrated in fig .1a with data for the novel _ howards end _ ( he in the following ) by e. m. forster taken from ref . where circles correspond to the raw data .the horizontal distribution for the largest -values means that only single unique words have the largest number of occurrences .the triangles corresponds to a -binning ( bin has a size of ) of the data and one notes that these data follow a smooth curve ._ this last fact implies that the data are produced by a stochastic process_. the functional form gives a good fit to the data ( in fig .the level of `` goodness '' of this fit is discussed in section iii and shown in fig . [ 4 ] .  instead of analyzing the complete book, one can analyze a _ section _ containing a total of words .then one finds that the `` power - law slope '' of the corresponding word - frequency distribution , , in a novel depends on the total number of words .this is illustrated in fig .1b , which shows the average word - frequency distribution for -parts of howards end .the total number of words is which means that the -part shown in fig .1b only has words while the -part corresponds to words .the word - frequency distribution for a section of size is obtained as an average over a large amount of sections of the same size and we use periodic boundary conditions in order to avoid reduced statistics due to the boundaries of the book . as will be shown below, real books display a strong tendency towards having the words close to randomly distributed , allowing for the use of periodic boundary conditions . as seen in fig .1b the slope of the power - law like part of the distribution gets systematically steeper when taking smaller and smaller sections of the book . from a practical point of viewthis means that if you attempt to approximate the word frequency distribution with the function then the exponent increases as decreases . _the change of the shape of _ _ as a function of the total number of words _ _ is a characteristic feature of the word frequency in a book_.  fig .2a shows the number of distinct words as a function of the total number of words : the first word is always distinct which means that .as you go further into the book , words tend to be repeated which means that the number of distinct words increases slower than a straight line with slope 1 .the shape of gives a characteristics of the novel since it reflects the spatial distribution of words within the novel .note that the function and the distributions are directly related , since the average number of times a distinct word appears is . how would changeif the words were completely randomly distributed in the book , keeping the same frequency distribution ? as seen from fig .2a , the function for the randomized book ( where all words are placed randomly in the book ) is very close to the raw data of the novel . _a characteristic feature of a novel is that the distribution is close to the one for the random null model of the novel_. this implies that the real novel and the null model share some overall random features .the random features are also reflected in the distribution of words belonging to different frequency classes : the frequency class contains all words which appears precisely times in the book .for example the class contains all the words which only occurs once in the book .random with respect to frequency classes means that there is no preference for words belonging to a specific frequency class to appear in any particular part in the book .thus for a random book you should have encountered close to half the words belonging to a frequency class when you have read precisely half of the book .2b shows the percentage of words belonging to a frequency class encountered after reading half a real book as a function of .the data is for the real he and the full drawn horizontal line is the expectation value for a randomized he .the grey shadings mark one and two standard deviations ( using the same binning as for the real novel ) away from the randomized he .this means that if the data circles in fig .2b had belonged to a single realization of a random he book then they would with large probability fall inside the grey areas .the actual circles in fig .2b give the data for the real novel he .these data follow the same horizontal trend and are compatible with the random null model over a substantial region of values .however a real novel is of course a highly purposely structured creation . some noticable deviations in fig .2b can immediately be associated with such contructive features . the first noticable deviation in fig .2b is that the value for the frequency class ( words which only occur once in the book ) is only 47% ( an average over the collection of books in appendix a gives 47,3% ) , which is a statistically significant deviation from 50% .the reason is that an author who writes a book from the beginning to the end will have a slightly decreasing tendency of introducing new rare words towards the end of the book .another noticable deviations is the two circles higher than 50% for larger ( words occurring very often in the book ) .these deviations are actually caused by the two specific words _she _ and _ her _ and are clearly context related features in the novel ( a particular context in chapter four about a third into the book has a very low concentration of _ she _ and _ her _ ) .nevertheless fig .2b illustrates that the overall tendency of the data has the same characteristic feature as the null model . for the simon model ,the distribution of words belonging to different frequency classes are incompatible with the random feature displayed by real novels : the triangles in fig .2b represent the data for a single simon book of the same length and as he .the dashed lines give the analytic asymptotic behavior in the small and large limits ( see appendix b ) .it is clear that rare words tend to appear very late in the simon - book while common words are more densely positioned early in the book .as explained below , this is because the simon model is a growth model .another characteristic feature of the null model is that the text is translationally invariant .this means that if you divide the novel into three consecutive sections and obtain the functions separately for all three sections then these three functions show no systematic trend in there deviations .2c demonstrates that the same is to very good approximation also true for the real novel he .appendix a gives data from a variety of novels suggesting that the qualitative agreements between the random null model and real novels given by figs .2a and c are indeed general features .real books contain information in the form of a story .different parts describe different events and surroundings which may creates word correlations .so , we should expect some fluctuations between curves for different parts of a novel .but the point is that , in general , no systematic change can be observed between parts of a real novel . _the translational invariance of the text is a characteristic feature of a novel . _whereas a real novel is in qualitative agreement with the null model , the _simon model _ is instead incompatible : fig .2d shows that the simon model does not obey translational invariance , but instead display a strong systematic trend .the data is obtained by generating books of the same length as he using the stochastical growth model by simon .the books are divided into three consecutive parts of equal size and the average distributions for these three parts are plotted in the figure . as seen the distributions systematically changes with the position in the book in a way that is incompatible with the translational invariance .this is contrary to the data for a real book ( compare fig .so what is wrong with the simon model in the context of real novels ? the problem can be traced to the stochastic element ( the dice ) in the model : the ground version of the _ simon model _ goes as follows : the novel is assumed to be written by adding words in a consecutive order from the start to the end . each time the author adds a word to the text it can either be a word not previously used in the text or an old word .there is a certain chance to add a new word and a certain chance to use an old one .the crucial stochastical assumption in the model is that the chance for picking a specific old word is directly proportional to the number of times this word has already been used in the text written so far .thus the randomness in the simon model is associated with picking words randomly from the text already written . as this text evolves , the reservoir ( the text written so far ) from which the random words are picked also changes .hence the random element in simon - type models explicitly depends on the growth process of the text .it means that the stochastic element changes with the position in the book .this is in contrast to the random null model , where the randomness is independent of the position in the book .one may also note that the resulting word - frequency distribution , , for the simon model , with a constant growth rate , is independent of the length of the text .this is in contrast to a real novel where the shape of the distribution changes with the length of the text ( compare fig .the crucial point is that stochastic text evolution models in general have the same problem as the simon model , including all preferential attachment type models   ._ growth processes which are based on a stochastic element ( a dice ) which ipso facto depends on the position in the text do not adequately reproduce the statistical distribution of words in a text ._ we emphasize that this is a fundamental structural feature which can not be remedied within this class of stochastic models .this implies that the stochastic element in real novels belong to an altogether different stochastic class .a noteworthy additional characteristic feature is that the word - frequency distribution for an author does to large extent only depend on the number of words written by the author and not on the specific book or short story .this is illustrated in fig . 3 by comparing a short story by d. h. lawrence ( _ the prussian officer _( po ) , ) with book sections of the corresponding size from two of his full novels . fig .3a is for _ woman in love _ ( wl ) which has and b ) for _ sons and lovers _ ( sl ) which has . as in the case of fig .1b , the word frequency distribution for a section is the average over many sections of the same size . in order to obtain a section size of the same length as the short story we use -parts in a ) and -parts in b ) .the agreement is very good in both cases except for the data of the very highest -values .this difference is an artifact of comparing a snapshot ( po ) with a curve resulting from averages ( sectioning of wl and sl ) .we now return to the characteristic size dependence of the word - frequency distribution for a novel described in fig .1b . in fig .4a compares this size dependence with the corresponding size dependence of the random null model : first we extract , directly from the raw data , the corresponding to sections -parts of the novel he .this data is represented by squares in fig .next we randomize the words in he .note that a randomization leaves the frequency distribution invariant . from a sample of the randomized he - book we extract corresponding to -parts of the randomized he .this is given by the triangles in fig .the overlap of the data is near perfect , indicating that the null model transform in very much the same way as the real novel . in case of the random null modelone can straighforwardly obtain the size transformation .the starting point is the word - frequency distribution for a book with total words and different words .the question is how relates to the word - frequency distribution , , for a section size of the very same book . for the casewhen the words within a frequency class are randomly distributed the relation follows from combinatorics .the probablility for a word that appears times in the full book to appear times in a smaller section ( ) can be expressed in binomial coefficients : if we let and be two column matrices with elements numerated by , then    where is the triangular matrix with the elements   and is the binomial coefficient .the coefficient is   since is a triangular matrix with only positive definite elements it also has an inverse which is given by    one should note that rbt ( random book transformation ) only hinges on the assumption that words belonging to a frequency class are randomly distributed through out the book .since this assumption is rather well obeyed by real novels ( compare fig .2b ) , the near perfect agreement between the randomized null model and the real he in case of the two -parts shown in fig .4a may be interpretated as a confirmation that the real novel and the randomized novel share some basic stochastical features .  in fig .4b we start from the randomized he and section it into parts with words . from each section size the average number of distinct words is determined so that one obtains the quantity .an average over many sections of the same size is used .the result is the full drawn curve in fig .one should note that this is in fact not a curve but a very dense set of data points ( each point corresponds to a different section size which means that the total number is ) . in this way the raw data for he given by the cirles in fig .1a are transformed into a very smooth curve for .the bayesean probabalistic assumption used is that words from different word - frequency classes have no preferential order . as apparent from fig .2b and fig .4a this is a very reasonable bayesean assumption .the point is now that the function through the rbt - transformation uniquely determines and vice versa .in order to find the corresponding we have used a parametrized ansatz for and determined the parameters so as to reproduce the -data as well as possible . in fig .4b we have tested three different parametrization forms . the first is a pure power law , , ( short dashed curve in fig .our conclusion is that a power law is incompatible with the data and can be ruled out .the next try is a power law with an exponential cut off , .this form gives a very resonable approximation of the data and the function representing the binned data in fig .1a corresponds to the long dashed curve in fig .but one can , off course , do a little bit better by adding another parameter .the augmentet power law with an exponential cut off , , gives an even better fit to the data ( open circles in fig .4b ) .  as simple quantitative goodness measure, one can take the maximum absolute difference between the real data and the data obtained from the various parametrizations : the values for the power - law , power - law with exponential cut off and the augmented power - law with exponential cut off are approximately , and , respectively . in fig .4a we have replotted the binned he - data from fig . 1a together with the best parametrization of obtained from the -data in fig .4b ( circles and dashed curve , respectively ) .the interesting point here is that our data analysis , which makes use of the rbt - transformation , makes it possible to distinguish between parametrizations of which would otherwise be very hard to distinguish .this is illustrated in fig .4c which directly compares the augemented power law with exponential cut off with the straight power law with exponential cut off .as seen from the fig .4c , there is almost no discernable difference when is plotted in a log - log scale .a consequence of the rbt - transformation is that the functional form of changes with the length of the text . the full drawn curve in fig .4a gives corresponding to -parts of he obtained from the parametrization of the form determined from fig .4b .it agrees very well with the real data .  in fig .3 it was demonstrated that the word frequency distribution , associated with -part sections of a novel of an author , to good approximation also describes a shorter novel by the same author , provided the shorter novel has the same length as the sections .one can then extrapolate this idea and imagine that the longer novel also can be described as a section of an even longer novel , and so on . this leads to the suggestion of a `` meta book '' , a giant single `` mother book '' which characterizes the word - frequency distribution of all the writings of an author .an author would then , when writing a novel , be roughly pulling a section of words from this `` meta book '' resulting in a word - frequency distribution .this is the same as transforming down the `` meta book '' via the rbt to the size .the `` meta book''-concept will be further explored in a forthcoming paper. have shown that the words belonging to a frequency - class in a book have a tendency to be randomly distributed thoughout the text .this randomness is incompatible with text growth models like the simon model .this is because these models are based on a stochastic assumption of re - using words already written in the text .this is true for all growth models , independent on the detail of the growth mechanism .it was also shown that the word - frequency distribution of a novel has a _ shape _ which systematically depends on the size of the novel .also this feature is incompatible the simon model .instead the properties of a novel were to large extent found to be shared with a random null model .the size transformation of this model is explicitly given by a random book transformation ( rbt ) and some consequences of this were explored .we speculate that the word - frequency is consistent with the concept of a `` meta book '' which characterizes the word - frequency distribution of all the writings of an author .our findings about the statistical properties of the words in a novel seem to be general : it does not matter much which author or book you pick , the overall properties are the same ( at least for the english novels we have so far analyzed ) .thus it does say something general about the structure of the written language used by a single author .since language in general is a product of the human evolution , it also means that the statistical properties presumably reflects some evolutionary pressure .this work was supported by the swedish research council through contract 50412501 . very helpful discussions with seung ki baekare also gratefully acknowledged ..list of the books analyzed . is the total number of words in the book , is the total number of _ different _ words in the book and is the average number of times a word is used .the initials of the authors stand for : e.m f e.m .h m herman melville .g o george orwell .t h thomas hardy .l d.h lawrence . [ cols="<,<,^,^,^",options="header " , ]   [ book_values ]  when compared to the simon - book , all the real books seem to have small values of and , indicating a strong resemblance to the null model of the random book .the values in the second row is also showing that there is no real trend among the real books , except for sl , which has a small negative trend ( compared to the simon - book which has a very strong _ positive _ trend ) .in the simon model a word is being written at every time step .with probability a new word , that has never been written in the book so far , is written . and with probability an old word is rewritten , chosen uniformly from the words existing in the book .this means that the probability for a word to be rewritten is proportional to the number of times it has already been written .when re - creating a real book the parameter ( ) is usually a small number ( ) and the length of the book ( ) is generally large ( ) .we want to start by calculating how big a fraction of a book , written by the simon - model , one has to read before having encountered half of all the words that appear only once in the book . to do this we need to calculate the probability that a specific word which is introduced at time is not repeated through out the book with length . at every time the probability for this word not to be rewritten is the sum of the probabilities that another of the words already written is rewritten ( ) and that instead a completely new word is written ( ) . at time , words have been written in total and words are still to be written , so the total probability becomes     since ( except for very small times , which includes only a tiny part of the whole text ) we make a taylor expansion around zero , approximate the sum with an integral and get ^t = \ln \rho^{t - t } + \frac{1}{\rho}\ln\left(\frac{t}{t}\right ) \label{b3}\end{aligned}\ ] ]   if we write a book , then is the average number of -words one gets from the introduction time , and so where is the total number of -words in the book .  {1/t}^1\nonumber\\  & = & \frac{t}{2-\alpha}\left(1-\underbrace{\left(\frac{1}{t}\right)^{2-\alpha}}_{\approx 0 } \right)\nonumber\\ \rightarrow w_d(1 ) & \approx & \frac{t}{2-\alpha}\end{aligned}\ ] ]  to find the time , , when we have introduced half of all the -words , we solve the expression : {1/t}^{t_{1/2}/t}\nonumber\\ & & = \left(\frac{t_{1/2}}{t } \right)^{2-\alpha}-\underbrace{\left(\frac{1}{t } \right)^{2-\alpha}}_{\approx 0 } \approx \left(\frac{t_{1/2}}{t } \right)^{2-\alpha}\nonumber\\ \rightarrow & & \frac{t_{1/2}}{t } = \left(\frac{1}{2}\right)^{\frac{1}{2-\alpha } } \label{t_1/2}\end{aligned}\ ] ] which is the fraction of the book one has to read before one half of the -words have been read . for the simon - book in fig . 2 ( ) this value is .that is , of the book .+ equation [ t_1/2 ] can be generalized into where is the fraction of one - degree words .+ next we want to do the same thing for -words .now we need to calculate the probability that if a word is first introduced at time it will only be repeated once at time .this probability is given by     again , this quantity gives the average number of -words one will get from words that are introduced at time and repeated at time , which means that where we sum over all possible combinations of and where .this can also be evaluated in a similar way as for the -case and we get     the first sum counts all the words where both its appearances happen before and is thus counted twice .the second sum counts all the words that was introduced before and repeated after and is thus counted as one .equation [ half_k2 ] can be evaluated into : equation [ t_1/2_k2 ] can not be solved analytically but a numerical solution for the simon - book in fig . 2 ( )gives the value .we now have two points ( and ) giving the asymptotic functional form for low k : s . in fig .2b a straight line was drawn intersecting these two point ( and ) to show this asymptotic behavior .the derivations for this quantity gets very complicated for larger values of since we are summing over all different words with the same frequency .but for very large :s we have words that are alone in their frequency - group .that is , they are the only one with that particularly frequency .this makes the derivation much simpler and we can get the asymptotic behavior for large :s . from ref . we get the equation   where is the number of occurrences a word will have in a book of length if it was introduced at time .we want to know at what time we have written half of those words .this is given by                     fig 1 : word frequency distribution for the book howards end ( he ) : a ) circles give the raw data .the horizontal tail reflects that the largest number of occurrences corresponds to single words .triangles give log - binned data and follow a smooth curve implying a stochastic origin .the actual data is to good approximation of the form with : b ) changes with the section size of the book .full curve represent the complete he , long - dashed curve and short - dashed curves represents sections corresponding to a 20th and 200th parts of he , respectively .the curves represent the log - binned data .fig 2 : number of distinct words as a function of the total number of words : a ) real and randomized he given by full and dashed curve , respectively .the close agreement implies that the words are close to randomly distributed throughout the book : b ) curves describing how big a fraction of the book one has to read before having encountered half of all the words with a specific frequancy .the circles and triangles represent the real he and a simon - book ( same size and as he ) respectively .the dashed lines are showing the analytic asymptotic behavior of the simon - book ( see appendix b ) .the full line represents the average result for a randomized book and the gray areas shows one and two standard deviations away from the random book .c ) for three different starting points within the book ; full , long - dashed and short - dashed curves correspond to the beginning , middle and end of he , respectively .the close agreement implies that the word distribution in a book is to good approximation translational invariant : d ) the same different starting points as in c ) assuming that the word - distribution was given by the simon text growth model .the large and systematic differences shows that the simon - type growth models do not describe the randomness of the word distribution in a real text .fig 3 : the sectioning of two full novels compared to a short story by the same author .a ) the circles represent the binned data of the full novel _ woman in love_. the triangles show the sectioning ( a 20th - part ) of the same book down to the same size as the short story _ the prussian officer _ , shown with squares .b ) the same as for a ) but for the full noval _ sons and lovers _ sectioned into an 18th - part .  fig 4 : the random book transformation ( rbt ) .a ) the data for he ( open circles ) is parametrized ( dashed curve ) .the dashed curve is transformed to a 200th - part of the book ( full curve ) .this full curve should correspond to a 200th - part of the randomized he ( open triangles ) .the agreement is striking .the distribution corresponding to a 200th part of the real he is given by the open squares .the close agreement with the triangles shows that the words are to large extent randomly distributed .b ) the function for he : full curve corresponds to the randomized he and the circles are obtained from the rbt using the parametrization of given in a ) .the agreement is perfect .the long - dashed curve corresponds to the data obtained from rbt using the parametrization of given in fig .1a and the inset , which is an in - zoomed version of the dashed squar , is showing how this curve is deviating from the real data . the short dashed curve in b ) represents a power - law fit to the word - frequancy distribution which clearly fails to represent the data .c ) is showing how similar the two parametrizations are which means that rbt determines to high accuracy .fig a1 : complementary figure to fig .2a and c showing the number of distinct words as a function of the total number of words for seven additional books : first column represents counting from start to finish and the second column represents counting through three consecutive parts of the same size .

Abstract: it is shown that a real novel shares many characteristic features with a null model in which the words are randomly distributed throughout the text . such a common feature is a certain translational invariance of the text . another is that the functional form of the word - frequency distribution of a novel depends on the length of the text in the same way as the null model . this means that an approximate power - law tail ascribed to the data will have an exponent which changes with the size of the text - section which is analyzed . a further consequence is that a novel can not be described by text - evolution models like the simon model . the size - transformation of a novel is found to be well described by a specific random book transformation . this size transformation in addition enables a more precise determination of the functional form of the word - frequency distribution . the implications of the results are discussed .


_ anomalous diffusion _ , where the associated underlying stochastic process is not brownian motion , has recently attracted considerable attention .this case is interesting because there may be long - range correlations , among other reasons .anomalous superdiffusion can be be related to fractional laplacian operators and/or so - called _ nonlocal _ operators , defined point - wise by their operation on a function as  dx ' .\label{eq : ell}\ ] ] the fractional laplacian is actually a special case of this equation .the present work will focus on these operators and in particular the associated stationary equation , analogous to the local elliptic equation .it should be noted that these nonlocal operators and the associated equations can be applied not only to problems of anomalous diffusion , but to a wide range of phenomena , including peridynamic models of continuum mechanics which allow crack nucleation and propagation .  in this workit will be assumed that the kernel appearing above is parametrized by a ( possibly infinite - dimensional ) parameter ; is assumed random .furthermore , some partial noisy observations of the solution will be available . in a probabilistic framework, a prior distribution is placed on and the posterior distribution results from conditioning on the observations .the joint distribution can often trivially be derived and evaluated in closed form for a given pair , as .hence , the posterior for a given observed value of can be evaluated , up to a normalizing constant .one aims to approximate _ quantities of interest _ for some , where for the forward problem and for the inverse problem . the likelihood is often concentrated in a small , possibly nonlinear , subspace of .this posterior concentration generically precludes the naive application of standard forward approximation algorithms independently to the numerator , , and denominator , .it is noted that , typically , will have to be approximated numerically , and this will lead to an approximate posterior density .monte carlo ( mc ) and sequential monte carlo ( smc ) methods are amongst the most widely used computational techniques in statistics , engineering , physics , finance and many other disciplines . in particular , if i.i.d .samples may be obtained , the mc sampler simply iterates this times and approximates .smc samplers are designed to approximate a sequence of probability distributions on a common space , whose densities are only known up - to a normalising constant .the method uses samples ( or particles ) that are generated in parallel , and are propagated with importance sampling ( often ) via markov chain monte carlo ( mcmc ) and resampling methods .several convergence results , as grows , have been proved ( see e.g.  ) . for problems which must first be approximated at finite resolution , as is the case in this article , and must subsequently be sampled from using mc - based methods , a multilevel ( ml ) framework may be used .this can potentially reduce the cost to obtain a given level of mean square error ( mse ) , relative to performing i.i.d .sampling from the approximate posterior at a given ( high ) resolution .a telescopic sum of successively refined approximation increments are estimated instead of a single highly resolved approximation .the convergence of the refined approximation increments allows one to balance the cost between levels , and optimally obtain a cost for mse .this is shown in the context of the models in this article .specifically , it is shown that the cost of mlmc , to provide a mse of , is less than i.i.d. sampling from the most accurate prior approximation .smc within the ml framework has been primarily developed in .these methodologies , some of which consider a similar context to this article , have been introduced where the ml approach is thought to be beneficial , but i.i.d .sampling is not possible .indeed , one often has to resort to advanced mcmc or smc methods to implement the ml identity ; see for instance for mcmc .smc for nonlocal problems however , is a very sensible framework to implement the ml identity , as it will approximate a sequence of related probabilities of increasing complexity ; in some cases , exact simulation from couples of the probabilities is not possible . as a result , smc is the computational framework which we will primarily pursue in this article .it is shown that the cost of mlsmc , to provide a mse of , is less than i.i.d .sampling from the most accurate posterior approximation .it is noted , however , that some developments both with regards to theoretical analysis and implentation of the algorithm , must be carefully considered in order to successfully use mlsmc for nonlocal models .this article is structured as follows . in section [ sec : theo ] the nonlocal elliptic equations are given , along with the bayesian inverse problem , and well - posedness of both are established . in particular , it is shown that the posterior has a well - defined radon - nikodym derivative w.r.t .  prior measure . in section [ sec : expec ] we consider how one can approximate expectations w.r.t .the prior and the posterior , specifically using mlmc and mlsmc methods .our complexity results are given also .section [ sec : num ] provides some numerical implementations of mlmc and mlsmc for the prior , and posterior , respectively .consider the following equation where is given by , the domain is simply connected , and its `` boundary '' is sufficiently regular and _ nonlocal _ , in the sense that it has non - zero volume in ,  under appropriate conditions on , , for , with only for the limiting local version in which ( or a similar uniformly elliptic form ) .following  denotes the volume constrained space of functions , and the fractional sobolev space is defined as follows , for , where we define for ease of notation .  for , the dual with respect to of some space , the weak formulation of can be defined as follows .integrate the equation against an arbitrary test function .now , find ( satisfying the boundary conditions ) such that where b(x , x ' ) dx ' v(x ) dx , \\f(v ) & : =  \int_{{\mathbb{r}}^d } b(x ) v(x ) dx . \end{split}\ ] ]   let denote the maximum diameter of an element of .the finite approximation of is stated as follows .identify some such that assuming the spaces are spanned by elements , then one can substitute the ansatz into the above equation , resulting in the finite linear system   in particular , the spaces will be comprised of discontinuous elements , so that the method described is a discontinuous galerkin finite element method ( fem ) .piecewise polynomial discontinuous element spaces are dense in for , and are therefore _ conforming _ when for , so there is no need to impose penalty terms at the boundaries as one must do for smoother problems in which the discontinuous elements are non - conforming .the following assumption will be made for simplicity   , with norm and inner product . is continuous with , and there exist such that for all  [ as : kernel ]  as shown in section 6 of , this implies that for all   * is continuous : ; * is continuous : ; * is coercive : for some .this actually follows from the poincar - type inequality derived in proposition 2.5 of .  hence , lax - milgram lemma ( , thm .1.1.3 ) can be invoked , guaranteeing existence of a unique solution such that in other words , the map is continuous .the system inherits solvability since the bilinear is a fortiori coercive on , so that   let be a parametrization of , where and , and either or compact. then the following theorem holds .if assumption [ as : kernel ] holds almost surely for and the map is continuous from to , then the map is almost surely continuous .hence for all , i.e. all moments exist . here denotes the space of random variables such that .[ thm : fwd ]  since assumption [ as : kernel ] holds almost surely for , then uniformly .so , the quantity of interest for all . therefore , for any such that , the result follows , since then for all .if assumption [ as : kernel ] holds almost surely for and the map is continuous from to , then the map through the _ discrete _ system is almost surely continuous and .[ thm : finfwd ]   for the inverse problem , let us assume that some data is given in the form where ^\top , \quad g_i\in v.  \label{eq : g}\ ] ] then the following theorem holds .the posterior distribution of is well - defined and takes the form with .[ thm : inv ]  the form of the posterior is obtained by a change of variables to } , which are independent by assumption . note the change of variables has jacobian 1 .so , changing variables back , this also gives the joint density .the posterior is obtained by normalizing for the observed value of .the form of the observation operator guarantees , where and is defined as in the proof of theorem [ thm : fwd ] , and is the matrix norm .  now define , \quad g_i\in l^2(\omega). ] : we define the joint probability distribution for the smc algorithm is if one considers one more step in the above procedure , that would deliver samples , a standard smc sampler estimate of the quantity of interest in ( [ eq : ml_approx ] ) is ; the earlier samples are discarded . within a multilevel context , a consistent smc estimate of is and this will be proven to be superior than the standard one , under assumptions .the relevant mse error decomposition here is : \}^2\big ] \le 2\,\mathbb{e}\big[\{\widehat{y}-\mathbb{e}_{\eta_l}[\varphi(\lambda)]\}^2\big ] +  2\,\ { \mathbb{e}_{\eta_l}[\varphi(\lambda ) ] - \mathbb{e}_{\eta_\infty}[\varphi(\lambda ) ]  \}^2\ .\ ] ]   we will now restate an analytical result from that controls the error term \}^2] ] , then convergence with respect to the norm is in fact .so , for linear elements , second order convergence can be obtained , leading to in .recall that [ as : kernel ] ensures well - posedness of the solution from , and so discontinuities are allowed .this is actually a strength of nonlocal models and one impetus for their use .it is reasonable to expect that if the the nodes of the discontinuous elements match up with the discontinuities , i.e. there is a node at each point of discontinuity for or there are nodes all along a curve / surface of discontinuity for , and if the solution is sufficiently smooth in the subdomains , then the rates of convergence should match those of the smooth subproblems .for example , if there is a point discontinuity such that the domain can be separated at that point into and one has and , where is the restriction of to the set , then one expects the convergence rate of to be preserved .this is postulated and verified numerically in .it is also illustrated numerically in that work that even with discontinuous elements , if there is no node at the discontinuity then the convergence rate reverts to , so in .the particular nonlocal model which will be of interest here is that in which the kernel is given by where , and is the characteristic function which takes the value 1 if and zero otherwise , and .notice that is scalar , but may be defined on either a finite - dimensional subspace of function - space , or in principle in the full infinite dimensional space .this may be of interest to incorporate spatial dependence of material properties .following we consider the following model .let  ] .for , let .the kernel is defined by , the following prior is used for the parameters , this is an extension of an example used in , which is identical to this model with and ( various values of are used there ) . to solve the equation with fem , a uniform partition on is used with , .thus for each level , . the same discontinuous galerkin fem as in is applied to solve the system .similar methods to those in are used to estimate the convergence rates for both the forward and inverse problems .the estimated rates are and .the rates and are used for the simulations .  for the inverse problem ,data is generated with , and .the observations are , where and ^\top$ ] .the quantity of interest for both the forward and inverse problem is .the forward problem is solved in the standard multilevel fashion by generating coupled particles from the prior at each level .the main result , the cost vs. mse of the estimates is shown in figure [ fig : mlmc ] . the bayesian inverse problem is also solved for this model , and the main result is shown in figure [ fig : mlsmc ] .this is the first systematic treatment of uq for nonlocal models , to the knowledge of the authors .natural extensions include obtaining rigorous convergence results for piecewise smooth solutions , exploring higher - dimensional examples , spatial parameters , time - dependent models , and parameters defined on non - compact spaces .* acknowledgements .* kjhl was supported by the darpa formulate project .aj & yz were supported by ministry of education acrf tier 2 grant , r-155 - 000 - 161 - 112 .we express our gratitude to marta delia , pablo seleson , and max gunzburger for useful discussions .   , f. , mazon , j. m. , jose , m. rossi , j. & toledo , j. ( 2009 ) . a nonlocal p - laplacian evolution equation with nonhomogeneous dirichlet boundary conditions ._ siam journal on mathematical analysis _ , * 40 * , 18151851 .

Abstract: this paper considers uncertainty quantification for an elliptic nonlocal equation . in particular , it is assumed that the parameters which define the kernel in the nonlocal operator are uncertain and a priori distributed according to a probability measure . it is shown that the induced probability measure on some quantities of interest arising from functionals of the solution to the equation with random inputs is well - defined ; as is the posterior distribution on parameters given observations . as the elliptic nonlocal equation can not be solved approximate posteriors are constructed . the multilevel monte carlo ( mlmc ) and multilevel sequential monte carlo ( mlsmc ) sampling algorithms are used for a priori and a posteriori estimation , respectively , of quantities of interest . these algorithms reduce the amount of work to estimate posterior expectations , for a given level of error , relative to monte carlo and i.i.d . sampling from the posterior at a given level of approximation of the solution of the elliptic nonlocal equation . +  * key words * : uncertainty quantification , multilevel monte carlo , sequential monte carlo , nonlocal equations , bayesian inverse problem + * ams subject classification * : 82c80 , 60k35 .


the analysis of the personal income distribution is one of the important subjects in the econophysics .the first step concerning this research is almost established .there are some empirical knowledge of it .it is believed that the distribution follows a power law in the high income range and follows a lognormal distribution in the low - middle income range .the purpose of this paper is to explain this behavior from a viewpoint of dynamical random surface ( drs ) model .the distribution in the high income range is first discussed by v. pareto in 1897 .the probability density function of the personal income is given by the following ; where is a normalization factor .also is the parameter which characterizes the profile of the income distribution , so - called pareto index , and this power law behavior is known as pareto law .on the other hand , it is indicated by r. gibrat in 1931 that the probability density function in the low - middle range follows a lognormal distribution as follows ; ,  \label{eq - gibrat}\end{aligned}\ ] ] where is a geometric mean value of and is a geometric variance .  in the early 80s , the above two knowledgeare rediscovered .the income distributions in united states 1935 - 1936 are analyzed in these papers .most of the data follows the lognormal distribution , while the higher 1% is governed by the power law .recently , this fact is reconfirmed by using the detailed japanese data .  the personal income distribution has been an interesting object of study .what seems to be lacking is a model from which we can deduce the features in the whole income range .for example there are models employed a stochastic evolution equation .it is easy to derive a power law , however , it is hard to lead a lognormal distribution by using these models . because the basic idea of these models is that they contain no typical scale in the fundamental equations .we should comment an exception . in recent paper ,the authors employed a concept of complex networks and a stochastic evolution equation with some interactions .as a consequence , they succeed to construct a power law and lognormal distribution in the whole range .there is , however , not an analytic consideration for creation of the lognormal distribution but only a phenomenological one by using computer simulations .our viewpoint is simple as follows .there is no typical scale concerning to money in the high income range . the distribution of it ,therefore , should obey a power law .on the contrary , there is a typical scale in the low - middle income range , and it causes the breaking of the power law distribution . as a toy model which can realize above two cases in the same framework , we employ drs model which is one of the methods for studying 2-dimensional quantum gravity with a typical scale .2-dimensional gravity without a term has no typical scale , unlike 4-dimensional gravity . here means the scalar curvature of a 2-dimensional surface .the dimension of is ( length) , namely the term is dimensionfull .the term , therefore , provides a typical scale to 2-dimensional gravity .it is well known that this model has two ranges . a fractal and self - similar structure , a power law ,is seen in the range where the typical scale does not affect the theory . on the other hand ,the fractal breaks down in the other range where the scale has the sense .this peculiarity is similar to the profile of the personal income distribution .  in order to confirm our view in the framework of drs model, it is straight to adopt the money itself as the fundamental variable of theory not the money possessed by individuals .we have once forgotten the concept of an individual and pay attention only to the connection of the money in a certain unit .this connection forms a 2-dimensional surface in drs model , and later the money of an individual is formed geometrically .the outline of this paper is as follows .section [ sec - minbu ] is a brief review of the formalism upon which our work is based . drs is explained and minbu analysis that is one of the methods for studying the property of random surface is described . the aspects of continuum quantum gravity in 2-dimensions and the two asymptotic forms of the partition function are presented in section [ sec - r2gin2d ] .these two forms correspond to the distribution functions in the high and the low income ranges , respectively .the interpretation and relation between quantities in drs model and ones in the personal income are discussed in section [ sec - map ] . in section[ sec - fitting ] , the practical fitting is shown .the personal income distributions of united states in the year 1935 - 1936 and japan in the year 1997 - 2000 are employed for its purpose .summary and discussion will be given in section [ sec - summary ] .in this section , we review drs and explain minbu analysis which is one of the methods for studying drs .drs model is known as dynamical triangulation ( dt ) in particle physics , and drs with a term has a fractal and a non - fractal ranges . in drs, we consider a 2-dimensional surface .this surface is constituted by attaching the sides of simplexes ( for instance equilateral triangles ) , and all possible constitutions are statistically averaged .the typical surface is shown in fig . [ fig:2-dim random surface ] .this surface has a self - similar structure .a simply connected region of area with one boundary is called a baby universe . in the dynamically triangulated surface ,a minimum neck between baby universes is composed of three sides of triangles and can be identified uniquely .the distribution of minimum neck baby universe ( minbu ) is , therefore , one of the important observable quantities in drs ( dt ) . in order to analyze it, we consider the case that a whole universe ( area ) is divided into two baby universes ( area , ) , paying attention to one of minimum necks ( fig .[ fig : divided minbs ] ) .we denote that is the fixed area partition function , and that is the partition function for surfaces ( area ) with one boundary ( length ) .minbu ( for minimum boundary loop length ) can be recognized as a surface from which one triangle is removed from a sphere .we can correlate these two partition functions , by counting ways which triangle is chosen and removed , as follows ; the partition functions of two minbus in fig .[ fig : divided minbs ] are given by we define , here , that is the statistical average number of minbus of fixed area on a closed 2-dimensional surface of area .if the 2-dimensional surfaces once divided into two minbus are made to be one universe again , the following expression can be obtained ; this formula works well as long as we treat minbus .it is well known that dt is equivalent to the continuum theory .if we take the continuum limit which brings every sides of triangles to the infinitesimal and the number of triangles to the infinity simultaneously keeping the area of a 2-dimensional surface constant , it is thought that this limit becomes equivalent to 2-dimensional gravity defined by the continuous variables .the explicit forms of the partition function and is obtained by using the knowledge of 2-dimensional continuum quantum gravity . in next section ,we consider it .2-dimensional quantum gravity with a term is investigated in the continuum framework in ref. .the partition function for fixed area is given by where is a metric , is a scalar matter field and is the number of scalar fields .also is the scalar curvature of the 2-dimensional surface .the -function fixes the area to .the actions are defined as where and are a bare cosmological constant and a coupling constant of length dimension , respectively .the actions ( [ action chi ] ) , ( [ action x ] ) and the integration measure in the partition function ( [ partition ] ) are invariant under 2-dimensional diffeomorphisms .the measure , therefore , should be divided by the volume of the diffeomorphisms , which is denoted by . the partition function ( [ partition ] ) carried out by an appropriate gauge fixing has the exact expression .the partition function , however , contains infinite excited modes and is difficult to integrate the full functional completely . in ref . , instead of the full integration , the two asymptotic forms are shown : where is a renormalized cosmological constant , and are string susceptibilities of 2-dimensional gravity .as for , the asymptotic form of the partition function ( [ infty ] ) represents that the surface is fractal .the similar phenomenon can be seen in 2-dimensional gravity without the term , where no typical length scale exists . in this range ,even if the model contains the term , at an area scale much larger than surfaces are smoothly fractal . on the other hand , as for , the partition function ( [ 0 ] ) is highly suppressed by an exponential factor  m^2x\rightarrow 0 m^2x\rightarrow 0 $ } ,  \label{eq - distribution - lowmiddle - cum}\end{aligned}\ ] ] where and are normalization factors .firstly , let us consider the distribution in the high income range . in our model ,pareto index takes the value of .we find that this index is almost consistent with the result of ( see u.s .data in fig .[ fig:1997 - 99-japan_1935 - 36-u.s . ] ) .also we should notice that this value is little larger than that in japan for the year 1997 - 2000 ( see the japanese data in figs .[ fig:2000-japan ] and [ fig:1997 - 99-japan_1935 - 36-u.s . ] ) .  secondly , the profile in `` the low '' income range is considered .this paper gives the weight to analyze this distribution .it must be noted that `` the low '' does not mean the low and middle .this point of view is one of the main differences from previous works .the plots of the data and the best fitting are also presented in figs .[ fig:2000-japan ] and [ fig:1997 - 99-japan_1935 - 36-u.s . ] .the parameter in the figures can be obtained as the slope in the graph where the horizontal axis is taken as and the vertical axis is taken as in the limit .  in our model ,the parameter specifies the aspect of the breaking of a power law , and we are able to make a best fitting by changing the value of it .the best fit values of in japan 1997 - 2000 and in u.s .1935 - 36 are given as , , , , and , respectively ( figs .[ fig:2000-japan ] and [ fig:1997 - 99-japan_1935 - 36-u.s . ] ) .we have employed the coefficient of determination for the parameter that indicates the appropriateness of practical fitting : here is the value of data , is the value of the fitting function and is the mean value of .it remains an unsettled question what distribution function should be employed in the middle income range .we have to inquire , to some extent , into this subject .it is believed that the distribution function is a lognormal one in this range .it is difficult to deduce a lognormal distribution from the partition function ( [ partition ] ) analytically .we employed , therefore , a computer simulation of dt with a term  to investigate a profile in the middle income range , is not so high neither low . here is a coupling constant and is the number of links at the vertex in drs .the result is shown in fig .[ fig : log_simu ] as the data of the distribution of minbus which is not cumulated .the horizontal and the vertical axes are the logarithm of and , respectively . as the figure indicates , the profile in the middle range can be well approximated with a secondary curve , and this behavior deduces a lognormal distribution .it seems that we can obtain the whole personal income distribution from the partition function ( [ partition ] ) .in this paper we have proposed the dynamical random surface ( drs ) model as a toy model to explain the profile of the personal income distribution .the money in a certain unit have been identified with a certain unit area ( simplex ) of a 2-dimensional surface .the connection of the unit money have been corresponded to the attaching of simplexes sides which forms a 2-dimensional surface . on the basis of this correspondence, we have analyzed the distribution of the 2-dimensional surface statistically as the personal income distribution by using minbu analysis .the point is that drs model has a typical scale , which is the coefficient of the term , and that the distribution of minbus exhibits two distinct phases .one is fractal in the large minbu range where the typical scale does not affect the drs , and the other is not fractal in the middle - small minbu range where small minbus are suppressed by the effect of the term . on the other hand ,this fact could be confirmed by the continuum theory which analytically presented the two asymptotic forms in these two region .moreover , we have proposed that the personal income should be corresponded to , where is the whole area of a 2-dimensional surface and is the area of a minbu . from this assumption ,the power law distribution has re - obtained in the high income range similarly to refs .in addition , it has deduced directly from the partition function in our method . at the same time, we have identified that pareto index , where is the parameter which characterizes the profile of the 2-dimensional surface in the limit , so called the string susceptibility .we have also concluded that we should chose the corresponding surface as a sphere without any matter fields ( ) .we have argued , in particular , the distribution functions in the low and the middle income range in detail .the new distribution function so - called weibull function has been proposed in the low income limit .this function has well fitted to the data rather than the lognormal distribution .a method of least squares has been employed to evaluate a validity of the data fitting .also we have employed the computer simulation to discussed the distribution function in the middle income range .our whole results were displayed in fig .[ fig:2000-japan ] and fig .[ fig:1997 - 99-japan_1935 - 36-u.s . ] .  in our argument, we assume that the personal income was identified with the area .we try to explain this identification below . in minbu analysis ,a boundary of a minbu is a minimum neck , where two universes are correlated through only three sides of simplexes each other ( see fig .[ fig : divided minbs ] ) .this means that the connection between universes is very weak at that point , therefore , it is natural to consider that the minimum neck separates the money which an individual possesses .we should notice that the area of a small minbu is double counted as the area of bigger one by definition .this may be because why the income which an individual obtains should be added to the income of those who have obtained more income . the effect can be neglect when is much smaller than .we must , however , take into account of it when is close to . in our model, there is the huge mother universe from which all minbus are removed ( see fig .[ fig:2-dim random surface ] ) .it is hard to find the object corresponding to this mother universe in the real economics , however , the theoretical distribution obtained has well expressed the real data .there may be virtual heat bath of the money in the real world .these arguments are left behind as a future subject .  in this paper, we have shown that the distribution of the personal income could be understandable as the geometry of a 2-dimensional surface of a sphere without no matter field .this suggests that the technique developed for the analysis of spacetime structure can be applied also to the analysis of our social structure . in 2-dimensions , above geometry is simplest one and the geometry of a 2-dimensional surface with genus and matter fields is also well understood .there are many fractal objects in the econophysics , for instance the distribution of the company profit , and the extended geometry may suit these objects .we are pleased if this analysis becomes a first step for the research which considers our society geometrically .the authors would like to express our gratitude to dr .w. souma , dr . y. fujiwara and professor h. terao for valuable advices and discussions .we are also grateful to professor h. kawai for useful comments especially about his work .thanks are also due to members of yitp , where one of the authors ( a. i. ) stayed several times during the completion of this work .one of the author ( t. s. ) is also grateful to professor y. yamagishi for useful discussions .mategna and h.e .stanley , an introduction to econophysics , ( cambrige univ .press , u. k. , 2000 ) .v. pareto , cours deconomique politique , ( macmillan , london , 1897 ) .r. gibrat , les inegalits economiques ( paris , sirey , 1931 ) . w.w .badger , mathematical models as a tool for the social science , ed .b. j. west , ( new york , gordon and breach , 1980 ) , pp .87 ; + e.w .montrll and m.f .shlesinger , j. stat .32 , 209 ( 1983 ) .h. aoyama , w. souma , y. nagahara , h.p .okazaki , h. takayasu and m. takayasu , fractals 8 , 293 ( 2000 ) ; + w. souma , to be appear in fractals 9 , ( 2001 ) , cond - mat/0011373 .the japanese tax administration , http://www.nta.go.jp .m. levy and s. solomon , int .c7 , 595 ( 1997 ) ; + h. takayasu , a.h . sato and m. takayasu ,79 , 966 ( 1997 ) ; + j. p. bouchaud and m. mezard , physica a282 , 536 ( 2000 ) ; + j.p .bouchaud , arxiv : cond - mat/0008103 ; + d. sornette , arxiv : cond - mat/0011088 . w. souma , y. fujiwara and h. aoyama , cond - mat/0108482 , kucp0189 .watts and s.h .strogatz , nature 393 , 440 ( 1998 ) ; + d.j .watts , samll worlds : the dynamics of networks between order and randomness .( princeton university press , princeton , new jersey 1999 ) ; + m.e .newman , cond - mat/0001118 ; + s.h .strogatz , nature 410 , 268 ( 2001 ) ; + r.albert and a .-barabasi , cond - mat/0106096 ; + s.n .dorogovtsev and j.f.f .mendes , cond - mat/0106144 .s. jain and s.d .mathur , phys .b286 , 239 ( 1992 ) ; + j. ambijorn , s. jain and g. thorleifsson , phys .b307 , 34 ( 1993 ) .kazakov , i.k .kostov and a.a .migdal , phys .66 , 2051 ( 1991 ) ; + j. ambjorn , b. durhuus and j. frohlich , nucl .b257 , 433 ( 1985 ) ; + f. david , nucl .b 257 , 543 ( 1985 ) .s. ichinose , n. tsuda and t. yukawa , int .a12 , 757 ( 1997 ) .h. kawai and r. nakayama , phys . lett .b306 , 224 ( 1993 ) .e. brezin , c. itzykson , g. parisi and j.b .zuber , commun .59 , 35 ( 1978 ) ; + f. david , mod . phys .a3 , 1651 ( 1988 ) ; + j. distler and h. kawai , nucl .b321 , 509 ( 1989 ) ; + v.g .knizhinik , a.m. polyakov and a.b .zamolodchikov , mod .a3 , 819 ( 1998 ) .e. brezin and v. kazakov , phys .b 236 , 144 ( 1990 ) ; + m. douglas and s. shenker , nucl . phys .b 335 , 635 ( 1990 ) ; + d.j . gross and a. migdal , phys .64 , 127 ( 1990 ) ; + m. douglas , phys .b. 238 , 176 ( 1990 ) ; + m. fukuma , h. kawai and r. nakayama , int . j. mod .a6 , 1385 ( 1991 ) ; commun .143 , 371 ( 1992 ) ; 148 , 101 ( 1992 ) ; + r. dijkgraaf , e. verlinde and h. verlinde , nucl . phys .b 348 , 435 ( 1991 ) ; + a. jevicki and t. yoneya , mod . phys . lett .a5 , 1615 ( 1990 ) ; + p. ginsparg , m. goulian , m. plesser and j. zinn - justin , nucl .b342 , 539 ( 1990 ) ; + t. yoneya , commun .144 , 623 ( 1992 ) ; int .a7 , 4015 ( 1992 ) ; + j. goeree , nucl .b358 , 737 ( 1991 ) .

Abstract: the results of dynamical random surface model ( 2-dimensional quantum gravity with a term ) are applied to explain the personal income distribution . a scale invariance exists if there is not the term in the action . the term provides a typical scale and breaks the scale invariance explicitly in the low and middle income range . a new distribution , weibull distribution , is deduced from the action analytically in the low income range , and a consistent fitting is obtained in the whole income range . also , we show that the lognormal distribution in the middle income range can be understood in this framework .