Principle of least effort vs maximum efficiency: deriving Zipf-Pareto laws

This paper provides a derivation of Zipf-Pareto laws directly from the principle of least effort. A probabilistic functional of efficiency is introduced as the consequence of an extension of the nonadditivity of the efficiency of thermodynamic engine to a large number of living agents assimilated to engines, all randomly distributed over their output. Application of the maximum calculus to this efficiency yields the Zipf-Pareto laws.


1) Introduction
Zipf's law is an empirical law describing the discrete distribution of some measured values x as a function of their rank r. If all the measured values are binned into W rank r in a decreasing order ( 1 > 2 > 3 … . > … > ), Zipf's law is given by [1][2] where α is a parameter to be determined experimentally. This regularity was first discovered by Estoup [3] and 30 years later by Zipf [1][2] who popularized it through more systematic and wider investigation of event frequency distribution. For language systems, represents the frequency of word in given text, being often close to one [2], but also possibly very different (larger or smaller) from unity in different language systems [4]. Other examples of rank distributions include, for instance, city size, where is population of a city with between 0.5 and unity [6], and individual income distribution, where is individual income with between 0.5 and 1.5 ( [2], [6] and references therein). In many cases, Zipfian law as a straightline with a unique slope in log-log plot is just an approximate representation of the decreasing tendency in the rank distribution [13][24] [25], better representations include, among many others, the Mandelbrot formula with one more adjustable parameter [18] and the generic models described in [13][24] [25] [26]. In my opinion, what is essential in all those observed Zipf or near-Zipf laws is the decreasing tendency of rank distributions, more or less close to a straight line in log-log plot, which must have some underlying universal mechanisms, being generic and common to all (living) systems considered.
Zipf gave a first interpretation of this decreasing rank distribution law by postulating that all human beings minimize effort in their activities to get some fulfillment [2]. This rule was first formulated in 1894 by Ferrero in his paper discussing the mental inertia of human being [5]. But Zipf was the first to explore its possible application to quantitative study. He wrote : "The power laws in linguistics and in other human systems reflect an economical rule: everything carried out by human being and other biological entities must be done with least effort (at least statistically)". This rule was obviously an intuition from the observation of the behaviors of human being himself and probably of other animals, always trying to get more done by doing less. A summary is given in [6] about several experimental works checking the rule of least effort.
The idea of least effort for human and animal systems is very appealing and even fascinating, especially in a perspective of using it for quantitative and analytic methods in the same way as many variational principles in physics (stationary action, least time, maximum entropy etc.). However, no such implementation with convincing mathematical calculus has been realized to date to our knowledge. The reader can find a summary of some of the efforts in [6], of which it is worth mentioning the work of Mandelbrot [18] and Canchu et al [21] who defined functionals of effort by considering information in language communication in order to apply the idea of least effort and to derive Zipf law for languages. Their work is an important step towards the implementation of the principle of least effort. I would only give two remarks here about the limits of their approach. The first is that the expressions of effort (cost) of [18], proportional to log of rank, is a little ad hoc, since it is obviously suggested in order for the variational calculus of maximum Shannon information to yield Zipfian power law, while the conventional outcome of this calculus is exponential distribution law. Another questionable point of [18] has been remarked by Manin [22] about the mutual dependence of the two parameters of the model which were thought to be independent. Concerning the proposition of Canchu et al [21], the functional of effort is proportional to the information communicated between speaker and hearer, while in a communication process, the information transferred should be the output of effort instead of effort itself. The effort in this case should be process dependent. If you are buying information, your effort is money. If you are searching for information in documents, your effort is related to time spent or to the volume of documents read. If you are trying to find information on internet, your effort can be estimated by the number of clicks made. If you are expressing information by speaking or writing, your effort is related to the number of different words employed or to the total number of words, and so on.
Obviously, the quantity of information you obtain is not necessarily proportional to your effort (money, time, clicks, number of words etc.). My second remark is that the approaches of [18] and [21] are limited to language and cannot be used to implement least effort for different living systems.
So the relationship between this beautiful principle and the Zipf's law remains a sort of speculation to date without solid and convincing mathematical proof. In the past several decades, much attempt was made to interpret or derive this power law with different mechanisms and models. There are almost as many models proposed as the systems in which Zipf's law and near-Zipf's laws have been observed. The reader can easily find relative information. The most recent models include, to our knowledge, about the origin of the Zipf's law in language by considering the interaction between syntax and semantics [7].
Another law we will focus on in this work is the Pareto distribution, a power law originally describing the wealth distribution of a population in a given society [8]: where X is a random variable representing the income, ( > ) the probability of finding a person with income larger than a value x, the smallest income and a constant characterizing the distribution. This distribution law is the origin of the famous 20-80 rule of Pareto. It is believed that there is an intrinsic link between Zipf's and Pareto's laws. If one is observed in a system, another must exist simultaneously [6] [19]. There is also an opposite view [20] which I will discuss later on in this paper The aim of this work is to derive these laws representing the decreasing tendency in the distributions, in a generic way from the idea of least effort for living systems or systems driven by living agents. For this purpose, a universal functional of effort is necessary for the minimum calculus of variation. However, an effort is a cost whose nature differs in different domains and processes. It can be an expenditure of energy, time, information, an amount of money and so on. It is difficult to define and quantify an effort in a general manner. In this work, I focus on another relative quantity instead: the efficiency. The notion of energy conversion efficiency is used throughout this work. This efficiency is defined by the ratio of the useful energy delivered by a system to the energy supplied to it over the same period of operation 1 . If a living agent, making effort for some achievement (output), is likened to the system delivering useful energy, its effort can be assimilated to the input (supplied) energy. We can then maximize the efficiency instead of minimizing effort. The maximum efficiency (MAXEFF) is obviously more advantageous and general than minimum effort, since, as defined above, efficiency involves both effort and output, its maximization implies not only minimum effort, but also maximum output which would be absent if the variational calculus of least effort was made with only a functional of effort. To my opinion, the true spirit of the principle of least effort can only be implemented by MAXEFF calculus.
The idea of MAXEFF in science and engineering is not new. A good example is the derivation of the Betz limit of the efficiency of wind turbine from fluid mechanics 2 [9]. The essential of the application of MAXEFF is to use an expression of efficiency as a functional. I adopt this method and introduce a functional of efficiency by considering the nonadditive property of the thermodynamic efficiency and the fact that we are tackling a large number of engines (living agents), all distributed over some output. Then the application of the maximum calculus to this functional of probability distribution yields the Zipf's and Pareto laws.

2) The Nonadditivity of efficiency
The definition of efficiency of a thermal engine in thermodynamics differs from one type of engines to another. For example, suppose an work engine absorbs an energy 1 , produces a useful work W, and rejects an energy 2 , in the ideal case without energy loss where all heat cost 1 − 2 is converted into work = 1 − 2 , the efficiency of this engine is defined by For a heat pump engine (conditioner for heating for example) which absorbs a heat 1 , consumes a work W, and produces a heat 2 for heating. We have = 2 − 1 if all work is converted into heat. Its efficiency is defined by = 2 = For a refrigerator (conditioner for cooling for example) which absorbs a heat 1 , consumes a work W, and rejects a heat 2 for cooling, we have = 2 − 1 if all work is converted into heat. Its efficiency is defined by  = 1 = The nonadditivity relationships of  and of In what follows, I only give a summary of the nonadditivity of  for two working engines.
Suppose two engines are connected in such a way that the first engine absorbs an energy 1 , does a work 1 , and rejects an energy 2 , and the second engine absorbs an energy 2 , does a work 2 , and rejects an energy 3 , one has  1 = 1 1 = If the engines cannot transform all the heat cost ( 1 − 2 = 1 for the first engine for example) due to energy loss (friction, vibration, heat conduction, heat radiation and so on), we can introduce a loss coefficient a in such a way to write (see Appendix I) Formally, = −1 corresponds to the case where the energy cost is totally converted into work.
If < −1, this is the case where the engines cannot transform all the heat cost into useful work, leading to a reduction of efficiency with respect to the case of = −1. On the contrary, if > −1, there is an enhancement of efficiency as if the collaboration created energy with respect to the = −1 case. This is possible only for processes in which the input and output are not energy or energy proportional quantities. This point will be discussed in the following section.
It is noteworthy that Eq.(7) takes place only for two engines with the output of one being the input of another. Different coupling between the engines gives rise to different additivity.
For example, if the second engine does not consume all the heat 2 rejected by the first engine, another parameter is necessary for describing this partial collaboration (see Appendix I-A2).
For two engines in contact with the same heat baths but having no exchange of energy, the total efficiency depends on the efficiency of each engine in a much more complicated way. In this work, we address living systems made up of coupled living agents having exchange of energy, information or other resources, making it possible for the output of ones to be the input of others. This property has been widely observed in most biological and social systems, at least those showing Zipf-Pareto laws [6]. The choice of the nonadditivity of Eq.(7) in this work is motivated by this consideration.

3) Modeling an ensemble of thermal engines
Suppose an ensemble of a large number of engines (or living agents). They are independent in the sense that there is no direct interaction or correlation between them, but their functioning s cooperative in the sense that the output of one engine can be used as the input of others and vise versa. This exchange of something (energy, information, entropy, money, even time) does not necessarily the existence of interactive forces between the agents. We do not consider the case where engines have no communication or collaboration between them. Each engine has a certain efficiency  with = 1,2 … . The total efficiency  of the ensemble should be a function of all  : Efficiency is in general nonadditive; hence, the total one  = ( 1 ,  2 …  ) cannot be a simple sum of the individual efficiencies. Based on the above analysis of the nonadditivity relationships for different type of engines (doing work, heating, cooling) and different collaboration type, we can model the whole system by using a simple nonadditivity given by Eq. (7) which also reads (1 + ) = (1 +  1 )(1 +  2 ). This equation can be written as, for the whole system of N engines: where is a parameter that characterizes collaboration between the engines as well as the energy loss during the processes from input to output. For many systems, can be free from energy connection for non-thermodynamic processes for which the input and the output quantities are not energy connected and there is no necessarily energy conservation condition.
For example, for living agents trying to be connected to some objects (sites, friends, cities, richness etc.), the output can be frequency of connection, the population or the agents' richness.
There is no energy conservation between these quantities and the inputs which can be energy cost, expenditure of time or money, used materials, number of actions and so forth. Another example is the economic process of investment. This process is similar to the process of a heat engine. The invested amount of money can be assimilated to the input heat 1 , the consumed input work is the effort to increase the profit, and the total turnover can be considered as the heat production 2 . In thermodynamics there would be a conservation condition: 2 − 1 = . But this relationship does not exist for the economic process of investment because there is no quantitative measure of the effort and of its conversion to 2 .
Eq. (8) is obviously the simplest model for the efficiency as a nonadditive quantity. The mathematical advantage of this model will be shown later. A little bit more complicated model can be where is the parameter characterizing the whole ensemble and is the parameter of the n th subsystem or agent. This composite model can be used when it is necessary to consider the composite effect of subsystems. In what follows, I focus on the ensemble as a whole, the one parameter model Eq.(7) or (8) will be used.

4) Efficiency as a functional of probability
Suppose that all agents in the ensemble are making effort to achieve as much as possible a measurable quantity represented by a variable having w discrete values with = 1,2, … .
More they get that quantity, larger is . This quantity can be income, wealth, city population, firm size, frequency of events, information quantity and so forth. At equilibrium (or stationary) state of the whole systems, all agents are distributed over the range of with agents at the value . We have ∑ =1 = . The probability of finding an agent at the value is = . The normalization condition is ∑ =1 = 1.
Due to the statistical nature of the model with a large number of agents distributed over all the values of X, it is reasonable to suppose that the total efficiency  of the agents that have the value depends on the number with  = ( ) or on the probability distribution  = ( ). The average efficiency  of the whole system reads  = ∑ =1  . Now let us separate the whole ensemble of agents into two independent subsystems A and B, with efficiency  ( ) and  ( ), respectively. The probability distribution of the agents in A is ( ) and that in B is ( ). The probability distribution of the whole ensemble can be written as We choose Eq. (7) as the efficiency nonadditivity. This implies a total efficiency given by It can be proved that Eq.(10) and Eq. (11) together Obviously  and  have the same functional of and , respectively. The proof of the unicity of Eq. (12) is shown in Appendix II.
The parameter b is related to a by the following considerations. First, due to the fact that the efficiency  is positive and that is smaller than unity, should have opposite sign of .
Secondly, from Eq. (11), if goes to the zero limit → 0, the efficiency tends to additive limit . This is to be proved analytically in the following section. . It is noteworthy that  is an monotonically increasing function of a in the interval 0 <  < ∞ for −∞ < < ∞.

5) Maximization of efficiency
As mentioned in the introduction, following the idea of least effort, I propose to maximize the efficiency, meaning that a calculus of variation applied to the functional of the average efficiency in Eq. (14) with respect to the probability ( = 1,2 … ). However, it is easy to verify that the maximization of  alone cannot lead to correct probability distribution. Indeed,  = 0 means  = 0 for all , leading to − = 0 meaning that = ∞ for > 0 and = 0 for < 0 . The introduction of normalization as a constraint of the variation, i.e., ( + ∑ ) = 0, will lead to uniform distribution = 1⁄ for all , which can be seen in The maximization of efficiency is not an isolated property. The efficiency is at its maximum when fulfillment is the best due to the effort of the living agents. Hence, there must a connection between the efficiency and the fulfillment or the output. It is quite reasonable to associate the maximum of efficiency to the maximum output. The output is represented by the variable X.
As a consequence, its average, ̅ = ∑ , should be maximized at the same time as the efficiency. As mentioned above, can be the income, the frequency of events, the population of cities, the size of companies representing their wealth and so forth. In the case of income for example, if the maximum total efficiency is achieved, then the least effort implies the total income of the population should reach its maximum as well. In other words, the two maximums are mutually conditioned through least effort. The functional to be maximized should be the sum ( + ̅ ). We write now where is a constant multiplier characterizing the balance between the two maximums.

7) Deriving Zipf's law
Now let us put the values of X into W bins. These bins are ranked in a decreasing order in the magnitude of . Let be the benchmark value of the bin of rank r, we have 1 > 2 …> >…> . The Zipf's law Eq.(1) describes the relationship between and r. As mentioned above, the Zipf's law and the Pareto law are regarded as two sides of the same thing and connected to each other by the fact that the cumulate probability ( > ) is proportional to rank r [19]. But there is also a point of view treating these power laws as two separate laws independent from each other [20]. Here we will show that these two laws are both consequence of MAXEFF and related to each other through another parameter.
By definition, the population (number of agents) having more income than increases with increasing . In other words, ( > ) increases when increases until its maximum at which X reaches its minimum value and ( > ) = 1. But saying that the probability ( > ) is always proportional to the rank r seems to be just an observation from some empirical results. It would be hard to say it is a general rule. In what follows, I will use MAXEFF to derive a general relationship between ( > ) and r leading to Zipf's law from  between and is not the same as = 1 obtained with the hypothesis of ( > ) ∝  . Clearly, the ZP efficiency increases with increasing G and is very large when G approaches its maximum (see Figure 3). This mathematical behavior of the ZP efficiency coincides with the classical point of view in economics [11]. Of course, in a real system, growth may deviate from this ideal behavior and undergoes the impacts of different factors. It is obviously unconvincing to say that an economic system is most efficient when only one hyper-trillionaire has all the wealth and the rest of the population has nothing, or that an educational system has maximal efficiency with only one person very educated and the rest of the population without any education (G=1). The relationship between wealth inequality and economic growth is an important issue and widely debated until nowadays 4 . In order to get more insight into the ZP efficiency, it is necessary to make further investigation of its relationship with other properties such as inequality, 80-20 rule, 20/20 ratio, performance, opportunity etc.) by either empirical study or numerical modeling of real systems [12]. Considering the universal character of the rule of least effort for living systems and that living agents can be likened to thermal engines in term of their efficiency, I hope this approach is meaningful to most, if not all, living systems and systems driven by a large number of living agents statistically obeying least effort and show power law behavior. As example of these systems, we can cite, among others, economic systems, linguistic systems, informational and social networks and so on. These are systems driven by the effort of human beings to achieve something. In economic system regarding income for example, income is the output people are looking for with effort. People must try in general to achieve more income by trying to do less effort (time, investment, physical effort etc.), leading to Pareto law in the distribution of income [8]. In linguistic systems, similar thing happens since people try not to be talkative (statistically, of course) and to express as much information and idea as possible by using as less words as possible. Here the output should be the quantity of information or ideas expressed, and the effort can be the time spent for writing or speaking or the total number of words as well as the number of different words used. The quantity of information and idea expressed is, of course, not easy to quantify and measure. However, this behavior of trying to express more by using less time and words tends to increase the frequency of individual word in a given text, as if the frequency of words was the thing (output) people are looking for with least effort, leading to Zipf law of frequency-rank distribution. Regarding the development of cities in an economic and political society, people and companies statistically tend to move to large cities for many reasons (financial, communication, education, comfot, entertainment etc.), as if the size of city was something they are trying to obtain with as less effort (time, money, physical effort) as possible, leading to Zipf-Pareto laws of size or distribution. The last example we think worth mentioning is the informational networks where the Zipf-Pareto laws is ubiquitous under the reign of preferential attachment [23]. This connection rule is a typical behavior of least effort or of maximum efficiency in the quest for information or other outputs. Its derivation from least effort will be presented in a coming paper.

9) Conclusion and remarks
We It is worth noticing that the result of MAXEFF in the present work is a single power law, or Zipf-Mandelbrot law, a near single power law. This is the objective of this work, to find a generic and system independent mechanism, in accordance with the principle of least effort,  [26]. It is possible to account for this behavior within this framework by considering an ensemble of agents, all differ in behavior as well as in the parameter a, in such a way that they can be reorganized into two or more subsystems with different parameter a.
This composite approach is to be developed in another work by using the technique of incomplete statistics [15]. It is also possible to explain the exponential distributions of certain complex systems [14] since the efficiency functional approaches logarithm form for small a.
The MAXEFF can generate near-exponential probability distribution in this case.
The resemblance of the ZP efficiency to Tsallis entropy 6 [15][16] is noteworthy. This comes from the fact that the ZP efficiency has the same nonadditivity Eq.(7) as Tsallis entropy.
However, for Tsallis entropy, this nonadditivity is a consequence of the postulated entropy [16].
But here, the nonadditivity, as suggested by the efficiency of thermal engines, is the starting point which uniquely leads to ZP efficiency. The parameter a has a concrete physical meaning as shown in the Appendix I. The maximization of Tsallis entropy has its origin in the Jaynes principle 7 [17] stipulating the maximization of Boltzmann entropy for thermodynamic systems, while the maximum efficiency arises from the principle of least effort for living systems which is well illustrated by the dictum "achieving more by doing less" that each of us is applying in every detail of our life.

Appendix I
Analysis of the nonadditivity of the efficiency of thermal engines.