Scoring Alternative Forecast Distributions: Completing the Kullback Distance Complex

We develop two surprising new results regarding the use of proper scoring rules for evaluating the predictive quality of two alternative sequential forecast distributions. Both of the proponents prefer to be awarded a score derived from the other's distribution rather than a score awarded on the basis of their own. A Pareto optimal exchange of their scoring outcomes provides the basis for a comparison of forecast quality that is preferred by both forecasters, and also evades a feature of arbitrariness inherent in using the forecasters' own achieved scores. The well-known Kullback divergence, used as a measure of information, is evaluated via the entropies in the two forecast distributions and the two cross-entropies between them. We show that Kullback's symmetric measure needs to be appended by three component measures if it is to characterise completely the information content of the two asserted probability forecasts. Two of these do not involve entropies at all. The resulting 'Kullback complex' supported by the 4-dimensional measure is isomorphic to an equivalent vector measure generated by the forecasters' expectations of their scores, each for one's own score and for the other's score. We foreshadow the results of a sophisticated application of the Pareto relative scoring procedure for actual sequentional observations, and we propose a standard format for evaluation.


Introduction
Bruno de Finetti's "partly-baked" musings with I. J. Good (1962) on the use of a penalty function to gauge the relative predictive qualities of competing forecast distributions have developed by now into a complete theory of proper scoring rules, replete with practical computational procedures. Gneiting and Raftery (2007) present a fairly comprehensive technical review of both the theory and its application. Lad (1996, Chapter 6) presents a pedagogical introduction. Proper scoring procedures are meant to provide the basis for a subjectivist understanding of forecast evaluations that can replace completely the common practice of "hypothesis testing." Such testing is regarded by subjectivists to be without foundation. The very notion that data are generated by an unobservable probability distribution is rejected in their characterisation of probability and its connection with empirical statistical research. In this present tribute to the imagination of de Finetti we propose a striking development in scoring theory. Our analysis gives rise to a completed understanding of Kullback's information measure in the context of the total logl scoring function. Implications for statistical practice are substantial.
The logarithmic scoring function for a probability mass vector (pmv) p N asserted by a forecaster "p" for a quantity X on the basis of an observation X = x o within its realm of possibility R(X ) = {x 1 , x 2 , ..., x N } is specified as the logarithm of the probability assessed for the outcome x o that is actually observed: S(X = x o , p N ) ≡ log(p o ). It has been known since early developments of proper scoring rules by Savage (1971) and by Matheson and Winkler (1976) that a pmv proponent's expectation of one's own log score equals the negentropy value embedded in that pmv. Symbolically, E p [log P(X = x o ))] = ∑ p i log p i . Moreover, p's expectation of the amount by which p's own log score will exceed the score of someone else's assertions, q N , is equal to the relative entropy in p N with respect to q N . That is, This relative entropy measurement is also widely known as the Kullback-Leibler (1951) directed divergence between the pmv's p N and q N , and is commonly denoted by D(p N ||q N ). A non-symmetric function of the two Subsequent consideration of scoring rules by Lad, Sanfilippo and Agrò (2012) has addressed the fact that the log score for mass functions, while proper, is incomplete in that it does not make use of all the observations with their asserted probabilities that are entailed in the assertion of p N and the observation of X . Augmented by the sum of (1 − p i ) log(1 − p i ) values for possibilities of X that are observed not to obtain, the log scoring rule becomes the total log scoring rule. Its prevision (expectation) according to the proponent who asserts p N equals the negentropy plus the negextropy of the asserted mass function. Negextropy was characterized in an article by Lad, Sanfilippo, and Agrò (2015) as a complementary dual of negentropy. The structure of the Kullback-Leibler divergence entertains a complementary dual as well: Parallel considerations define the complementary symmetric Kullback distance as D c (p||q) ≡ D c (p||q) + D c (q||p). Together, the Kullback distance and its complementary distance sum to the Kullback total symmetric distance: The present article presents two new developments in our understanding of associated issues. In the first place, we consider the expectations of the asserters of both p N and q N , each for one's own total log score and for the other's score. Attention turns to these proponents' considerations of the net gains they each might achieve on the basis of their professed willingness to trade their achieved scores for their personally assessed prevision values. (This constitutes the very definition of their previsions.) We find an appropriate strategy for assessing the relative merits of the two pmv assertions: perhaps surprisingly, it is to award to each forecast proponent the net gain achieved by the other's pmv, scaled appropriately, rather than to award to each forecaster one's own achieved proper score.
Secondly, our analysis turns to the pair of expected net gains and expected comparative gains of the forecasters. We learn that Kullback's total symmetric distance measure can be generated equivalently by two other pairs of functions that are different from that which specifies the Kullback-Leibler divergence. This result suggests that Kullback's symmetric distance function should be extended to a four-dimensional vector function if it is to characterise all the distinct features of the information expressed in the two pmv's. It turns out that the vector space spanned by this 4-D vector function is isomorphic to the space spanned by the two proponents' previsions for their own scores and for each other's score.
Section 2 of this article identifies the two pmv proponents' expected net gains and expected comparative gains according to their uncertain assessments of an unknown quantity X . Section 3 develops the logic of the Pareto-optimal assessment of their comparative forecast quality we propose here, while the scale of this comparative scoring procedure is discussed in Section 4. Scaling requires consideration of the variances assessed by the two forecasters for various score components. Attention turns to the higher dimensions of the Kullback complex that arise from these considerations in Section 5, and Section 6 displays why the appended dimensions of the complex are not Bregman divergences. Section 7 portrays the historical context of Kullback's analysis of the problem, and outlines its reconsideration according to the subjectivist understanding. Section 8 presents briefly some experience with and some proposals for application to statistical problems.
Deliberations in this article make use of the syntax preferred for Bruno de Finetti's operational subjective construction of coherent prevision, discussed by de Finetti (1967de Finetti ( , 1972 and Lad (1996, pp. 39-43). Events are characterised as numerical quantities (numbers, not sets) with the realm of possibility R(E) = {0, 1}. Otherwise they are addressed with the same syntax as any other quantity whose realm is finite and discrete. Prevision, P(X ), is a personal value assertion about X , a price at which its proponent is avowedly indifferent to a prize of X or of P(X ). Since a prevision is an assertion made by someone, the P symbol shall be subscripted by a mark denoting who is making the assertion, as in P p ( . ) and P q ( . ) used here. We shall refer to the proponents of these previsions merely as "p" and "q", respectively. One's prevision for an event is also called a probability. Parentheses around an expression that may be true or false denotes an event whose numerical value equals 1 if the expression is true, and 0 if it is false. Thus, for example, E i ≡ (X = x i ) is an event that equals 1 if and only if it happens that X is in fact found to equal x i ; otherwise E i = 0. In the more commonly used measure-theoretic syntax of formalist probability ala Kolmogorov, prevision is equivalent to expectation; in these terms, the expectation of a 0 − 1 random variable is a probability.

Interpersonal assessment of the Total Log Scores
The purpose of this Section is to formalise the notation for the total logarithmic score that will be used to assess our two forecasting distributions (pmvs), and to introduce the formalities of two concepts that will be central to our considerations. The first is the net gain achieved by a forecaster who asserts a pmv for an observable quantity. It defined as the difference between the score that will be achieved and the forecaster's expectation for the score. The second is the comparative gain achieved by one of the forecaster's score relative to that achieved by the other. It is defined simply as the difference between their two scores. We can then identify each forecaster's professed expectation for each of these gains. In doing so we will have identified four fundamental previsions that are relevant to evaluating the forecast quality of the two forecast pmv's on the basis of observations.
Consider the standard problem of statistical forecasting an unknown quantity X whose realm of possible observation values is R(X ) = {x 1 , x 2 , ..., x N }. An associated vector of partition events, E N , is composed of its logically related components E i ≡ (X = x i ) for i = 1, 2, ..., N. The sum of these events must equal 1 since one and only one of them must equal 1. Because the considered value of X is unknown when the forecasters assess probabilities, it is just not known which of the events is the 1 and which of them are the 0's. This will be identified when we observe the value of X . We shall study the uncertain opinions of two people who publicly assert distinct probability mass functions via the vectors p N and q N , respectively. Once the value of X is observed, their assertions will be assessed via a widely touted proper score, the Total Log Scoring function, defined for p N (and similarly for q N ) by as long as the pmv p N is strictly positive and N > 1. Here p o denotes the probability asserted for the value of (X = x o ) that eventually comes to be observed; and the sum in the second line of (1) includes all summands of the The "total log score" is a negative valued score, but it constitutes an award. A larger score (smaller absolute value) is achieved by a more informative distribution. As extremes, the pmv that accords probability 1 to the value of X that occurs, (X = x o ), receives a score of 0; those which accord probability 1 to any particular value of X that does not occur receive −∞. The uniform pmv receives log(1/N) N.B. From this point on we shall no longer write subscripts on vector quantities to denote their size, nor the limits on the range of summations, always understood to be summations over units from 1 to N.
To foreshadow a relevant consideration that will be addressed at the very end of the analysis, we note here that each of the two summands constituting the total log score, ∑ p log p and ∑(1 − p) log(1 − p) is itself a proper score of the pmv p, as is any positive linear combination of these two. Whereas the constructions we now discuss are developed in terms of the total log score, they pertain to each of these addends separately in the same way.

Previsions for one's own score and for another's score
In the following discussion we shall denote the entropy and extropy functions, respectively, by Sums such as − ∑ p log(q) and − ∑ q log(p) are called "cross entropies", and will be denoted by CH(p, q) and CH(q, p). Similarly, "cross extropies" will be denoted by CJ(p, q) ≡ − ∑(1 − p) log(1 − q). Recognise that cross entropies/extropies are not symmetric in their arguments. For example, CH(p, q) = CH(q, p).
Theorem 1. The linearity of coherent prevision identifies the previsions of both p and q for their own scores and for the other's score as: The proof is immediate from the application of linear coherent previsions to the first line of equation (1).
These four previsions exhaust the content of what p and q assert about their own prospective score values and for the scores to be achieved by one another, before the value of X is observed. We shall designate them as the four fundamental previsions, and we shall learn why this nomenclature is appropriate. Let's think a moment about their relative sizes.
Since the scoring function S T L ( . , . ) is proper, each of p and q expects to achieve a greater score than will the other; for the proponent of a pmv expects to achieve a maximum score by publicly professing one's actual pmv, p, rather than any other pmv, q. For this reason, the use of proper scoring rules is said to promote honesty and accuracy in the profession of probabilities. Thus, for example, −[H(p) + J(p)] surely exceeds −[CH(p, q) + CJ(p, q)], a statement which is true when p and q exchange places as well. However, the relative sizes of the two assessors' own expected scores, P p [S T L (X , p)] and P q [S T L (X , q)], may be in any order. This just depends on what they each assert about X , which is p or q, respectively.
When cross-score previsions are considered, p's prevision for q's score may be either smaller or larger than q's expectation for q's own score. In a computational survey of pmv's within the unit-simplex we found that the size of P p [S(X , q)] exceeds P q [S(X , q)] in about 10 to 15 percent of paired (p 3 , q 3 ) selected uniformly at random from the unit simplex. We have not yet characterised the situations in which this occurs, though it should be straightforward.
N.B. From this point we no longer use bold print to distinguish vectors from their components. These will be identifiable by their context.
We are now ready to analyse important features of p's and q's expectations regarding one another's scoring performance.

Expected Net Gains and Expected Comparative Scores
Consider first the "net gains" to be achieved by the proponents of p and q as a result of their receiving the total log scores, S(X , p) or S(X , q) respectively, as a gain in return for paying out the prices at which each of them values this score: (2) Both p and q expect a personal net gain of 0, since their assertions are presumed to be coherent. One's prevised (expected) score is the price at which a person values this unknown score to be achieved. The proponent of a pmv is avowedly willing both to buy and to sell a claim to the score for this price. For example, p is avowedly indifferent to the values of S(X , p) and However, the proponents of p and q do not expect each others' net gains to equal 0. Before assessing them, it is worthwhile to recognize that there is something arbitrary about our definition of Net Gain. For the forecaster is also indifferent to an award of the negative net gain, the result of an exchange in which the forecaster offers to sell the value of S(X , p) in return for receiving P p [S(X , p)]. This would reverse the sign of an awarded Net Gain as we have defined it. This is the sense in which an award to p of a proper score value or its prevision assessment would be arbitrary. Nonetheless, we are now ready to study it as defined.
Theorem 2. P p [NG(X , q)] may be positive or negative valued, as may P q [NG(X , p)].
As proof, note specifically that Now we have already noticed and remarked that p's prevision for q's score may be greater than or less than q's prevision for the same. If p expects a score for q greater than q expects, then p will expect a positive net gain for q because p is thinking that q expects too low a score. In such a case, If p expects a score for q less than q expects, the reverse is be true: The forecasters' expectations of their comparative gains relative to the other's are another matter. Defining the "comparative gain of p over q" on the basis of observing X as it is easy to see that both forecasters expect a positive comparative gain.
Theorem 3. Both forecasters always expect a positive comparative gain for themselves.
As proof, it is clear that and similarly, using the four fundamental previsions we identified in Theorem 1. Both of these previsions are positive because the scoring rule is proper. (3) and (5), it is apparent that summing each of the paired expected net gains and comparative gains yields precisely the negative results of one another. It is worth stating this too as a theorem.

Now comparing the equation pairs numbered
Notice however that the paired summands for these opposite magnitudes are different.
We shall now continue this analysis by devising an improved procedure for the method of comparative scoring, and then we shall use this result to extend our understanding of Kullback's symmetric divergence.

Evaluating pmv's by trading inhering Net Gains
Since p assesses the personal net gain NG(X , p) with prevision 0 while valuing NG(X , q) with the non-zero prevision specified in equation (3), p would surely be willing to trade a claim to plus or minus NG(X , p) in exchange for a claim to NG(X , q) if P p [NG(X , q)] is positive, and to −NG(X , q) if that prevision is negative. For p assesses NG(X , p) with value 0, whereas either NG(X , q) or −NG(X , q) is assessed with positive value. In the same way, q values claims to NG(X , q) with prevision 0, while valuing the net gain of p with the prevision P q [NG(X , p)], as specified in in the lower half of equation (3). Thus both p and q would be pleased to make an exchange in which p receives (appropriately either the positive or negative) value of the NG(X , q) from q while q receives similarly the value of the net gain NG(X , p) (or its negative) from p. In this exchange, both p and q would each be providing the other with something personally regarded as worthless. Thus, both of them would be pleased by the exchanges. Altogether they constitute a "Pareto optimal" exchange.
To clarify this waffle about receiving "a net gain or its negative", note for example that q is avowedly willing to buy or to sell the value of S(X , q) for P q [S(X , q)], since q values these equally. Now with these two options available from q as proclaimed, p would willingly take up q's offer to to sell S(X , q) at this price if P p [S(X , q)] > P q [S(X , q)], or to take up q's offer to buy S(X , q) at this price if P p [S(X , q)] < P q [S(X , q)]. The same structure of relative preference applies to q in reacting to p's proclaimed indifference to S(X , p) and P p [S(X , p)]. To simplify the formalities of this waffling about the action whether to buy or to sell, during further discussion we shall denote the content of such exchanges by saying NG * (X , q) is exchanged for NG * (X , p). This may mean that p gives NG(X , p) to q in return for P p [NG(X , p)] or that p gives P p [NG(X , p)] to q in exchange for NG(X , p), at the choice of q. Simultaneously, q gives to p the value of NG(X , q) in return for P q [NG(X , q)] or vice versa at the choice of p. Of course this specificity would need to be recognized and recorded in the computation of the resulting net gains.
One difference in the assessed values of the two sides of such an exchange is that both p and q would assess their own net gains being given up with a different variance than the net gains they would receive. This constitutes no problem for the usual characterisation of coherent prevision nor for the acceptability of the proposed exchanges. In defining prevision, the scales of exchange are overtly kept small enough that linearity of utility applies to practical valuations. However, we can rescale these agreeable exchanges so that both p and q would regard the two sides of their exchanges symmetrically with the same variance, that is, the same risk. To facilitate this redesign, the pmv proponents would want to rescale the two sides of the exchanges they offer by their assessed standard deviations for them. That is, p would offer to give q NG * (x,p) Both p and q regard both of these offers with positive value, so both would be accepted.
Thus, we have found an intriguing and novel solution to the question of how to evaluate the relative quality of p's and q's pmv assertions when the value of X is actually observed. Simplifying the net result of those two exchanges that are agreeable to both of them, the appropriate way to score the pmv assertions of p and q would be to award to q the score of ,p)] , and to award to p the score of These exchanges constitute for us a Pareto optimal scoring procedure for comparing the relative quality of p's and q's forecasting distributions : to award to q the amount of p's achieved Net Gain * scaled by the sum of the square roots of p's and q's precisions for that Net Gain * , and to award to p the amount of q's Net Gain * scaled by the sum of the square roots of p's and q's precisions for that Net Gain * .
In order to compute these relative information scores, we need to compute four relative variances: Let us digress momentarily to think about these variances in a brief Section of its own.

... and as to the variances
Here are some considerations regarding the cross assessments of the standard deviations. Firstly, remember that NG(X , p)] is a specified number. Now think of the total log score in the form of S(X , p) = log( p o 1−p o ) + ∑ log(1 − p) identified in the third line of equation (1). In this form it is clear that These follow for the same reason, that the summations ∑ log(1 − p) and ∑ log(1 − q) are specified numbers.
according to the standard result for any quantity, that V (Z) = P(Z 2 ) − [P(Z)] 2 .

Kullback's Symmetric Divergence Complex
In this Section we shall learn how and why the structure of Kullback's symmetric divergence function naturally suggests that it be embellished by vectorial components in three independent dimensions.
To begin, recall the details of the Kullback-Leibler directed divergence function D( . || . ) and its extropic complement D c ( . || . ). These functions are also known as the relative entropy and extropy functions of p with respect to q: and similarly for the "reverse directions", Those strict exceedance inequalities presume only that the pmvs p, q are not identical: p = q. Kullback's (1959, p.6) symmetric divergence function D(p, q) is generated by summing the non-symmetric directed divergences: (q||p). Similarly for the complementary symmetric divergence, D c (p, q) ≡ D c (p||q) + D c (q||p). The "total symmetric divergence" is denoted by D T (p, q) = D(p, q) + D c (p, q) .

Expected comparative gains sum to the total divergence
Now an intriguing idea derives from examining once again the expected comparative gain equations, this time algebraically in the context of the Total Log Scoring function, and recognising their relations to the Kullback-Leibler directed divergence functions. We state this as a Corollary 1 to Theorem 3. In the context of the Total Log Scoring function S T L (X , p) introduced in equation (1), the previsions for the comparative gain reduce algebraically by direct substitutions to Each pmv proponent's expected comparative gain over the other's is equal to the relative entropy plus the relative extropy of the proponent's own pmv with respect to the pmv of the other. This is the Kullback-Leibler directed distance, D( . || . ), plus its complementary dual, D c ( . || . ), both of which are known to be positive. So both the proponents of p and q expect a positive comparative gain in total score relative to the other. The positivity of each's expected gain over the other does not depend on which of them expects to receive a larger score personally. Again, both of them expect to receive a larger score than the other (on account of the propriety of the scoring rule) but surely the size of only one of their own actual scores will be larger than that of the other. In fact, their actual comparative gains are the negative equivalents of one another, according to their defining equation (4).
It is now worth noting that the sum of p's and q's previsions for their comparative gains can be seen in equations (5) to equal the total symmetric divergence of the pmv's.
Corollary 2 to Theorem 3. Again in the case of the Total Log Score, This is despite the fact that the sum of their actual comparative gains equals 0. Further investigation yields still another intriguing equivalence.

Expected Net Gains sum to the same thing
Intrigue arises from examining now the algebraic details of p's and q's expectations of each others' net gains, viz., , and similarly Each of these is the difference between one's own expection and the other's expectation of the other's log odds for the occuring event.
Now summing these expected net gains yields a companion to Corollary 2 of Theorem 3, evident from basic definitions: Corollary 3 of Theorem 3. Again using the Total Log Scoring rule,

An alternative generator of D T (p||q).
Perhaps surprisingly, at this late stage in the study of the divergence function, we can now recognise that still two more companion functions can be identified whose sums yield the same symmetric divergence as do the Kullback-Leibler functions D(p||q) and D(q||p). The awareness arises from Corollaries 2 and 3 of Theorem 3. The first pair, also directed functions, arises formally from switching the components of the finite discrete entropy function, −H(p) = ∑ p log p that get "differenced" in the production of D( . || . ). Rather than thinking about ∑ p (log p − log q), we consider differencing the "p" instead of the "log p", raising consideration of ∑(p − q) log p. We shall denote this alternative directed function by ∆( . || . ), motivating it below: and then similarly for the "reverse directions", Referring to our formulation of cross prevision assertions, it is apparent that ∆(p||q) = CH(q, p) − H(p), the amount by which q's cross entropy for p exceeds the entropy in p itself. Similarly, ∆ c (q||p) = CJ(q, p) − J(p), the amount by which q's cross extropy for q exceeds q's own extropy.
These considerations yield a surprising result: both the directed distance D(p||q) and its reverse D(q||p), and this new function ∆(p||q) and its reverse ∆(q||p), sum to the same thing, ... Kullback's symmetric divergence, which we are denoting by D(p||q). The same is true of the complementary sums, which we denote by D c (p||q). This result arises as Corollary 1 to Theorem 4.
Here the sums of the ∆( . || . ) functions apply the total log score to the Net Gains, while the D( . || . ) functions arise from its application to the Comparative Gains.

Identifying still a third function that yields D T (p, q)
Expanding the identical sums D(p||q) + D(q||p) and ∆(p||q) + ∆(q||p) which constitute D(p, q) into their component pieces, and then exhibiting them across a line makes it apparent that each of the functions, D(p||q) and ∆(p||q) arises by selecting two of the four summands to define the function, and thus the remaining two to define their "reverses", D(q||p) and ∆(q||p).

D(p||q)
is the sum of the first two pieces of the first line, leaving D(q||p) composed of the second two; while ∆(p||q) is the sum of the first and third piece, leaving ∆(q||p) to be composed of the second and fourth piece. Now seeing these sums and these function pairs in this way suggests that a third (and the accompanying sixth) choice of two of the four summands may be worthy of study as well. Consider using the interior two summands (the second and third pieces of the first line) to constitute still another form of function that would sum (along with the remaining pair) to yield the symmetric divergence D(p||q). This choice would be the sum of the "cross entropy" functions, CH(p||q) +CH(q||p). One difference of this function from D(p||q) and ∆(p||q) is that this sum of cross entropies is already symmetric on its own, whereas the latter two functions require summation with their companions D(q||p) and ∆(q||p) to construct a symmetric function. A second difference of the sum CH(p||q) +CH(q||p) from the D(p||q) and ∆(p||q) functions appears in its location for self-divergence. Whereas the values D(p||p) = ∆(p||p) = 0, the value of CH(p||p) +CH(q||q) = H(p) + H(q), the summed entropies in p and in q. However, defining the symmetric divergence function more generally than heretofore so to eliminate this location shift of "self-distance" yields a pleasing symmetrical result.
Theorem 5. Based on a generalised definition of symmetric divergence, so to account for the location of "self-divergence", Of course those self-divergence values in the first two lines equal 0, whereas in the third line CH(p||p) = H(p) and CH(q||q) = H(q).
The same procedures of deconstruction and reconstruction can be performed with the complementary functions (D c ( . || . ) and ∆ c ( . || . )) in a second line: D c (p||q) is the sum of the first two pieces of this second line, while ∆ c (p||q) is the sum of the first and third pieces.
We propose that the Kullback directed distance function needs to be appended by all three of its three generators if it is to characterise the information content of the statistical problem the forecasters address. Together they constitute a complex.
One might be puzzled. "Who cares about all these generators of the same symmetric distance? The symmetric distance is what it is." The answer derives from an identifiable linear relation between the four functions D T (p, q), D(p||q) + D c (p||q), ∆(p||q) + ∆ c (p||q), CH(p||q) +CJ c (p||q) and the "four fundamental previsions" we introduced in equation (2). We address this now.

The isomorphism of the Kullback complex of generators with the four fundamental previsions
Each of these eight component functions in a Kullback complex is a different linear combination of the various entropies, extropies, cross entropies and cross extropies that have been central to our scoring analysis. These combinations happen to be ordered in such a way that the two spaces spanned by the 4-dimensional vector functions are isomorphic, related by the linear equations What this relation tells us is that each of the three distance function generating pairs measures a distinct information source for understanding of the full information content of Kullback's symmetric distance. The symmetric distance measure is incomplete on its own: it needs to be supplemented by three more components if it is to represent the information content of the pmv assertions p and q. In all, it is identifiable better as a vector which we might refer to as the Kullback complex ... [D(p, q), D(p||q), ∆(p||q),CH(p||q) +CH(q||p)], rather than merely by the symmetric function value D(p, q) alone.
One further recognition is in order. We have couched this discussion in the context of the total logarithmic score (1) and the Kullback total symmetric distance function D T (p||q). It is worth remarking that the total log score can be considered to be the sum of two separate components: the log score itself, ∑ E i log p i , and the complementary log score, ∑Ẽi log (1 − p i ). Actually, both of these components of the total log score are proper scores on their own, as is any linear combination of them. The two component scores assess different aspects of the asserted pmv's p and q. One piece focuses more directly on the center of the mass function, while the other focuses more on the tails. Because these aspects of a pmv may concern different people in different ways with respect to utility, it is usually best to keep track of the two components of the score separately. If the scoring function S T L (X , p N ) were considered to be the sum of the two separate scores, say, S log (X , p N ) + S complog (X , p N ), the isomorphism equations (9) hold separately for each component of the total log score and their previsions too. Tracking these components separately, the total symmetric Kullback complex could be considered to be an 8-dimensional space of measurement functions.
We shall now address a final technical note concerning the relation of these newly recognised dimensions of the information complex to the theory of Bregman divergences.