in geosciences , principal component analysis ( pca ) has been an essential and powerful tool at detecting spatial structures amongst time series recorded at different locations .pca aims at building a decorrelated representation of the data and it finds the spatial patterns responsible for the largest proportion of variability .pca can also be viewed as a dimensionality reduction technique that extracts the most relevant components of data , i.e. the ones that maximize the variances .hence , second order moments are the foundation of pca . but relying exclusively on second moments implies that pca is only optimal when applied to multivariate gaussian vectors .although rarely stated and even more rarely checked , this underlined normality assumption is not always satisfied in practice .recently , different approaches have been tested to extend the applicability of pca in geosciences .in particular , nonlinear pca ( nlpca ) has been applied to several geophysical datasets ( e.g. , * ? ? ? * ; * ? ? ? * ) . in the nlpca algorithm ,data are considered as the input of an auto - associative neural network with five layers , with a bottleneck in the third layer . through the minimization of a cost function ,the output is forced to be as close as possible to the input , and the bottleneck layer is a low dimensional representation of the input .since this neural network is nonlinear , nlpca goes automatically beyond correlations . however , nlpca suffers the intrinsic limitations of multilayered networks ( e.g. , * ? ? ?* ; * ? ? ?* ) : it is computationally expensive and does not always converge to a global solution . to overcome these difficulties ,we pursue a less ambitious aim : instead of trying to find decompositions that can explain the entire body of data with respect to a criterion , we focus on the part of data responsible for large anomalous behaviors .in contrast to pca , our approach tends to give maximal weight to data points which largely deviate from the mean , and to find the corresponding representative spatial patterns , i.e. the directions along which such points are prominently distributed .the key element in our procedure is the expansion of the cumulant function that can provide information beyond the first two moments . by maximizing the cumulant function over growing hyperspheres in the data space , a set of componentscan be derived .we first apply our procedure to three types of multivariate random vectors ( normal , skew - normal , gamma ) .then we demonstrate in the general case that if a direction exists for which the marginal probability density of projected data display a larger tail than in all other directions , then our procedure is able to select that direction .  when finding the first component , we show that pca is a special case of our approach whenever the gaussian assumption is satisfied . besides the gaussian case we show that , for any probability density , the solution derived from the centered cumulant function can be transformed to the first principal component by decreasing the radius of the hypersphere .other principal components could be found as well , by a generalization of the proposed method .our method is computationally cheap , and the solutions are found in the form of unit ( normalized ) vectors , as in the case of pca , allowing a unidimensional projection with an easy geometrical interpretation .  in summary, this paper focuses on the problem of characterizing spatial patterns associated to large anomalies , i.e. large deviations from the sample mean , when the data set under study can not be assumed to be normally distributed .in the univariate case , the cumulant function of the random variable with finite moments is defined as the following scalar function \big\}= \sum_{n=1}^{\infty}\kappa_n  { s^n \over n!},\ ] ] where , represents the mean function and the scalar corresponds to the cumulant of .the first two cumulants and are simply the mean and the variance of , respectively .the third and fourth cumulants are classically called the skewness and the kurtosis parameters . concerning the existence of cumulants , we assume in this paper that all the cumulant coefficients are finite and that the cumulant function is always well defined . the cumulant function and its coefficientshave many interesting properties .for example , if and are two independent random variables then the cumulant of the sum is equal to the sum of the cumulant of and the cumulant of for any integers .if follows a gaussian distribution , then all but the first two cumulants are equal to zero . in a multivariate framework , the cumulant function of the random vector is simply defined as \big\ } \mbox { , for all  .}\ ] ] as in the univariate case , the linear and gaussian properties associated to the cumulant function defined by ( [ eq : g def ] ) still hold , but the cumulant coefficients formulas are more cumbersome to write down in a multivariate framework . for more information about cumulants , we refer the reader to .  to identify possible favorite projection directions with respect to the multivariate cumulant function , we first rewrite the vector in ( [ eq : g def ] ) as the product , where , the scalar represents the norm ( radius " ) of and is the unit angular / direction " vector defined as ( note that ) .secondly , the cumulant function for our vector projected along the direction vector is introduced as \big\ } , \nonumber\\ & = & \sum_{n=1}^\infty k_n(\mbox{\boldmath \(\theta\)})\frac { |{\mathbf s}|^n}{n!}.\end{aligned}\ ] ] our algorithmic strategy is to maximize the cumulant function at fixed non - small with respect to the angular component that varies over an unit hypersphere .practically , we have to find the optimal directions defined by , \mbox { for any .}\ ] ] if the radius is small enough , and the mean is zero , the variance dominates the centered cumulant function and the contributions of other cumulants can be neglected .in this situation , finding the first pca component can be viewed as a special case of this optimization procedure , because maximizing the cumulant function for small is equivalent to maximize the variance .as the value of grows , higher and higher order cumulants become more dominant .our main goal is to find in ( [ eq : max pb ] ) for the largest admissible and study their properties .we will call the solutions of such an optimization scheme the _ maxima of the cumulant function _ ( mcf ) directions .we anticipate that , since the scalar product is invariant under orthogonal transformations , the cumulant function is invariant as well .given that the unit hypersphere is also invariant , our algorithm is symmetric respect to orthogonal transformations .for instance , if data vectors are rotated by a given angle , the solutions of the algorithm , in terms of , are rotated by the same amount .this symmetry implies that if the probability density is isotropic , then the cumulant function is isotropic as well , and no relative maxima of exist on the unit hypersphere . in that caseno directions are selected : for a rotationally symmetric distribution there is indeed no preferred direction along which anomalies are prominent , they are distributed uniformly over all angles .  to illustrate our optimization procedure we derive explicit cumulant maximization schemes for three special cases of multivariate family distributions ,see section [ sec : theory ] . for assessing the outputs of our algorithmwhen applied to real data , we refer the reader to the second part of this paper ) .to study the properties of the maximization method developed in section [ sec : cumul fct ] , three examples of distribution functions are considered in this paper .we choose these three families because explicit results can be derived and they have been classically used in the statistical modeling of temperatures and precipitation data .  without loss of generality ,some relevant matrices are diagonal thereafter . however , since the solutions are invariant respect to orthogonal transformations , they may be rotated along with the corresponding coordinate change .analytical calculations are performed for the general multivariate case , while figures are given for the bivariate case .suppose that the data at hand can be appropriately fitted by a multivariate gaussian vector .we assume that the observations have been centered ( zero ) mean and we denote the covariance matrix as .the cumulant function of the centered gaussian vector ( e.g. , * ? ? ?* ) is equal to \big\ } =  { 1 \over 2 } { \bf s}^t \sigma { \bf s }  .\ ] ] hence , it is easy to show that all cumulants but the second are equal to zero .the decomposition in equation ( [ eq : projg def ] ) , , implies that the cumulant function becomes our optimization problem is to maximize under the constraint . to find the optimal defined by ( [ eq : max pb ] ), we introduce a function to be maximized , constrained by the lagrange multiplier , as setting the gradient with respect to to zero gives let be the largest eigenvalue of the covariance matrix and its associated eigenvector . introducing , we can write . consequently , is the solution of ( [ eq : eigen ] ) .note that depends on but not on .the optimal direction , for the gaussian case , is , and it corresponds to the classical first principal component .  to illustrate this result , a bivariate vector of the normal distribution is presented in figure [ nfig ] ( in which a contour plot is drawn in logarithmic scale ) . the matrix is assumed to be diagonal , with entries 1.2 and 0.5143 .   and ( entries of the diagonal covariance matrix ). the first principal component is shown ( pc1 ) , together with the two ( opposite ) maxima of the cumulant function ( mcf1 and mcf2 ) .all vectors are in arbitrary scale .the maxima of cumulant function are parallel to the first principal component , all pointing towards the large anomalies , in terms of high probability ( at fixed vector norm ) .probability contours are ,,  the first principal component ( pc1 ) , corresponding to the eigenvalue 1.2 is horizontal , while the second principal component , corresponding to the eigenvalue 0.5143 is vertical , and is not displayed .the two maxima of the cumulant function ( mcf1 and mcf2 ) are just the positive and negative part of pc1 . indeed , both and are solutions of ( [ eq : eigen ] ) .while this is always true for pca , the maxima of the cumulant function may in general neither be parallel nor orthogonal .pc1 is indeed the direction along which large anomalies are ditributed in the gaussian case . in order to derive pc2 and other higher principal components from the cumulant function, one would have to determine not only its maxima , but also its minima and saddle points .   to introduce skewness to the gaussian density , while keeping some of valuable properties of the normal distribution , azzalini and his co - authors ( e.g * ? ? ?* ; * ? ? ?* ; * ? ? ?* ) have extended the normal density to a larger class called the skew - normal ( sn ) density , that is defined as where is a multivariate normal probability density function with zero mean and covariance matrix , and is the cumulative density function of an univariate gaussian random variable with zero mean and unit variance .the vector corresponds to the degree of skewness .when , there is no skewness , and the sn distribution reduces to the gaussian case . from ( [ sndist ] ), it is possible to derive the cumulant function ( e.g * ? ? ?* ) of a sn vector \big\ } =  { 1 \over 2 } { \bf s}^t \sigma { \bf s } + \log \left ( 2 \phi({\scriptstyle \sqrt{\frac{\pi}{2 } } }  \mbox{\boldmath }^t { \bf s } ) \right).\ ] ] where represents the mean vector , and is equal to note that the covariance matrix of the sn distribution is not but  .a bivariate example of the sn distribution is presented in figure [ snfig ] , in which a contour plot is drawn in logarithmic scale .the matrix is chosen to be diagonal with entries and .the skewness vector is taken to be equal to .the distribution in figure [ snfig ] has been centered such that the mean vector is zero . in the bivariate example, is .   andthe matrix is chosen to be diagonal with entries and .the first principal component is shown ( pc1 ) , together with the two maxima of the cumulant function ( mcf1 and mcf2 ) .all vectors are in arbitrary scale . in this case , the two maxima of the cumulant function are not related to the first principal component , and point towards the ( local ) large anomalies , in terms of high probability ( at fixed , and large , vector norm ) .probability contours are ,,  from the sn cumulant function , we can write the cumulant function of the centered vector ) as \ ] ] while it is not possible to find explicit solutions of the maximization problem defined by ( [ eq : max pb ] ) for ( [ genfunsn ] ) , one can provide valuable approximated solutions for both small and large . in the former case , the following two taylor expansions are the key elements to derive our results  where erf corresponds to the error function defined by + then we can write the following approximation   & = & \mbox{log}\big[1+\mbox{erf}({\scriptstyle \frac{\sqrt{\pi}}{2}}|{\mathbf  s}|\mbox{\boldmath }^t\mbox{\boldmath \(\theta\)})\big ]  \\  &  =  & \mbox{log}\big[1+|{\mathbf  s}|\mbox{\boldmath }^t\mbox{\boldmath \(\theta\)}+o(|{\mathbf  s}|^3)\big ] \mbox { , by ( \ref{log(1+s ) } ) } , \\  & = & |{\mathbf  s}|\mbox{\boldmath }^t\mbox{\boldmath \(\theta\)}-{|{\mathbf s}|^2\over 2}\mbox{\boldmath \(\theta\)}^t\mbox{\boldmath }\mbox{\boldmath  }^t\mbox{\boldmath \(\theta\)}+o(|{\mathbf s}|^3 ) \mbox { , by ( \ref{erf})}.\end{aligned}\ ] ] from equation ( [ genfunsn ] ) , it follows that the cumulant function is approximately equal to as previously noticed , the matrix represents the covariance of the sn distribution .hence , the maximization of the right hand side of ( [ genfunsnsmalls ] ) is equivalent to solving the system defined by ( [ eq : eigen ] ) , but instead of working with , we just need to replace by in ( [ eq : eigen ] ) .consequently , the solution to maximize the sn cumulant function in the neighborhood of zero is the largest eigenvector of the matrix , i.e. the pc1 of the sn covariance matrix .  for large , this result does not hold and different directions are obtained .we need to recall the asymptotic expansion of the error function & \mbox { , as , } \\         2  & \mbox { ,  as . }     \end{array }      \right.\ ] ] if , the logarithm in equation ( [ genfunsn ] ) can be expanded as   & = & \mbox{log}\big[1+\mbox{erf}({\scriptstyle \frac{\sqrt{\pi}}{2}}|{\mathbf  s}|\mbox{\boldmath }^t\mbox{\boldmath \(\theta\)})\big]\\ & \simeq & -\frac{\pi}{2}\frac{|{\mathbf  s}|^2}{2}\mbox{\boldmath \(\theta\)}^t \mbox{\boldmath }\mbox{\boldmath }^t\mbox{\boldmath \(\theta\)}+o\big(\mbox{log}(|{\mathbf s}|^{-1})\big).\end{aligned}\ ] ] in this case , we have the direction that maximizes is again a eigenvector , but this time is the eigevector of the matrix , pointing towards .  if , we have \simeq   \left\ { \begin{array}{ll }   \log 2 & \mbox { , \mu},\\   0 & \mbox { , if \mu}.\\  \end{array } \right.\ ] ] then , the cumulant function can be approximated by and maximizes by the largest eigenvector of , pointing towards .  in summary ,depending on the size of ( small or large ) and the sign ( positive or negative ) , the solutions of equation ( [ eq : max pb ] ) for the sn distribution can be viewed as the largest eigenvectors of three different matrices , , and .  for the bivariate example of figure [ snfig ] , the pc1 is shown ( in arbitrary scale ) , explaining 60% of the variance .the second pc ( 40% of the variance ) is orthogonal to pc1 and is not displayed .both maxima of the cumulant function for large , denoted as mcf1 and mcf2 ( respectively for and ) , are presented in figure [ snfig ] for the bivariate example ( same scale as pc1 ) .the two local maxima point towards the large anomalies of the distribution : this can be seen by noting that a point at the upper - right end of pc1 corresponds to a small probability , and hence is less likely to be found , than a point at the right end of mcf1 ( the two points being of equal norm ) . similarly , a point at the down - left end of pc1 is less likely , in probability , than a point at the down end of mcf2 .   this section investigates the multivariate gamma distribution defined by cheriyan and ramabhadran ( see * ? ? ?each component of the data vector is distributed following a gamma distribution , and the components depend each other by means of an auxiliary variable .the joint distribution is where is a gamma distribution , i.e. for , equal to zero otherwise .a bivariate example is presented in figure [ gfig ] , with , and in ( [ gampdf ] ) .  , , .the first principal component is shown ( pc1 ) , together with the maximum of the cumulant function ( mcf1 ) .all vectors are in arbitrary scale . again, the maximum of the cumulant function is different from the first principal component , and points towards the large anomalies , in terms of high probability ( at fixed , and large , vector norm ) .probability contours are ,,  the cumulant function for this multivariate gamma distribution can be written as \big\}=-\alpha_0\mbox{log}\big(1-\sum_{i=1}^n s_i\big )   - \sum_{i=1}^n\alpha_i\mbox{log}(1-s_i)\ ] ] and the mean of the component is . by replacing by , the cumulant function of the centered vector can be written as for small , the logarithms are approximated by the truncated taylor expansions , i.e. and it follows that the cumulant function can be approximated by where the covariance matrix is defined by  hence , for small , the solution of ( [ eq : max pb ] ) is the classical pc1 eigenvector of the covariance matrix associated with the largest eigenvalue .it is plotted in figure [ gfig ] for the bivariate example .the negative part of pc1 is not displayed , as well as the second principal component which is just orthogonal to the first .  for large , the cumulant functionis not defined , since the logarithms in equation ( [ genfungam ] ) must have positive arguments , for all unit vectors . in particular, the following two inequalities have to be satisfied however , we take the largest allowed value of , which is considered as a valid limit .since is a unit vector , the maximum of each component is , which holds when all the other components are zero , while the maximum for the sum is , which holds when .the largest allowed value of is then : in that case remains finite for all s , except for , where it diverges due to the first logarithm of equation ( [ genfungam ] ) ( all the others remain finite ) . for larger values of , diverges over subspaces larger than a single point .hence the boundary case is taken as representative for a `` large '' limit , and the point is taken as the maximum of the cumulant function .for the bivariate example , the maximum is plotted in figure [ gfig ] , denoted as mcf1 , and scaled to pc1 .note that for high values of the probability distribution ( e.g. the smallest contour ) , pc1 seems a representative direction of the egg - like shape of the distribution , but for low probabilities it becomes clear that the line is responsible for the large deviations .a point at the end of pc1 is indeed less probable than a point at the end of mcf1 .this result is understood by noting that the joint density ( [ gampdf ] ) is the probability distribution of variables defined as , where the variables are independent and gamma distributed with parameters .hence , a large deviation of , which occurs independently on others s , corresponds to a large deviation of which is placed on average along the line ( that corresponds to for the cumulant function ) .we have seen that the multivariate cumulant function reduces to the variance if the radius tends to zero . in that case, its maxima corresponds to the first principal component of data set .if grows , higher order cumulants come into play , but is not clear what the corresponding maxima represent . in order to clarify this point , we rewrite the cumulant function defined by ( [ eq : projg def ] ) in terms of the explicit integral over the probability density this expression can be reduced to an unidimensional integral , by defining the projected data as , and the marginal probability density of the projected data , .the cumulant function is then    which corresponds to the cumulant function of the univariate vector with distribution density . in the light of this representation ,our maximization procedure is better understood : we are looking for directions that correspond to a marginal probability density displaying maximal cumulant function , at fixed .we want to demonstrate that if grows , a larger cumulant function corresponds to a marginal density with a fatter tail .hence our procedure selects the directions corresponding to the marginal densities with fatter tails , where the anomalous behaviour is expected .specifically , consider two different directions and : we want to demonstrate that if the marginal distribution along has a fatter tail than the distribution along , then the cumulant function has also a fatter tail along respect to .more formally , we have the following theorem .let and be two directions .if there exists a real such that the density distribution of the random variable is strictly larger than the density for all , i.e. then there exists a radius such the cumulant function of and satisfies   in order to prove the result , we start by noting that the following inequality holds >  \exp(|{\mathbf s}|z^*)\big[f_{\mbox{\boldmath \(\theta\)}}(z)-f_{\mbox{\boldmath \(\theta\)}'}(z)\big]\ ] ] for all , where we have replaced the exponential function with its minimum value in the interval . the inequality holds because the density difference is positive in this interval , by assumption .since the above inequality holds in the whole interval , it can be integrated over , i.e. dz > \\ \;\;\;\;\;\ ; \exp(|{\mathbf s}|z^*)\int_{z^*}^{+\infty } \big[f_{\mbox{\boldmath \(\theta\)}}(z)-f_{\mbox{\boldmath \(\theta\)}'}(z)\big ] dz.\end{aligned}\ ] ] given that the two densities are normalized , i.e.   we can rewrite the right hand side ( r.h.s . ) of the last inequality by inverting the integration order and the marginal densities as dz=\nonumber \\\exp(|{\mathbf s}|z^*)\int_{-\infty}^{z^*}\big[f_{\mbox{\boldmath \(\theta\)}'}(z)-f_{\mbox{\boldmath \(\theta\)}}(z)\big ] dz\nonumber\end{aligned}\ ] ] rearranging the four terms gives indeed the normalization condition times the exponential .note that since the integral in the left hand side ( l.h.s . )is , by assumption , positive , the integral in the r.h.s . must be positive as well , because they are multiplied by the same positive constant .  given the above inequalities , we only need to demonstrate that it exists an such that , for all , dz > \\\int_{-\infty}^{z^*}\exp(|{\mathbf s}|z)\big[f_{\mbox{\boldmath \(\theta\)}'}(z)-f_{\mbox{\boldmath \(\theta\)}}(z)\big ] dz\nonumber \end{aligned}\ ] ] where the value of must be determined .note that even if the integral in the l.h.s .is positive , the density difference is not guaranteed to be positive for all .if the integrand was positive as well , equation ( [ in3 ] ) would hold trivially for all values of , because the exponential in the l.h.s .is always larger than that of r.h.s .this corresponds to the case in which , beside , we assume also .however , we do not need this additional request , and we just note that it would help our procedure by leaving the problem of using a large .  since the exponential is positive , we can rewrite equation ( [ in3 ] ) as dz > \\\int_{-\infty}^{z^*}\exp\big(|{\mathbf s}|(z - z^*)\big)\big[f_{\mbox{\boldmath \(\theta\)}'}(z)-f_{\mbox{\boldmath \(\theta\)}}(z)\big ] dz\end{aligned}\ ] ] for all .the integral in the l.h.s .is positive , as stated above , and is independent on , while the integral in the r.h.s .converges to zero for , as long as the density difference remains finite , because the exponential tends to zero in the whole interval .if we define as the largest possible value of for which the two integrals are equal , then for all the integral in the l.h.s . is larger than that of r.h.s .finally , we have for all  > \\ \int_{-\infty}^{z^*}dz\;\exp(|{\mathbf s}|z)\big[f_{\mbox{\boldmath \(\theta\)}'}(z)-f_{\mbox{\boldmath \(\theta\)}}(z)\big].\end{aligned}\ ] ] by rearranging terms , this is equivalent to , and the theorem is demonstrated .in this paper , we have introduced a novel method selecting the spatial patterns representative for the large deviations in the dataset .the method consists in finding the vectors in the space of data for which the cumulant function is maximal . as in the case of pca ,the spatial patterns are found as normalized directions in the space of data , and a linear projection can be performed , with an easy geometrical interpretation .if one is interested on the large deviations , the projection allows to safely perform extreme value analysis ( ) . in both cases the subspacesare ordered : in pca the order follows the fraction of variance of each subspace ; the maxima of the cumulant function are ordered by the value of . however , while pca accounts for the mass of the distribution , the cumulant function can cover the full range of the distribution .principal components are always symmetric , while large anomalous patterns , if generated by nonlinear processes , are expected to be neither specular nor orthogonal .accordingly , the maxima of the cumulant function are not necessarily symmetric , since they account for the whole structure of dependencies , and not only covariances .other nonsymmetric techniques , such as oblique varimax rotations , generally depend on the frame of reference : vector solutions do not covary with the space of data under orthogonal transformations .hence , solutions depend not only on the shape of the underlying probability distribution , but also on its orientation : an undesirable property for our purposes . the maxima of the cumulant function , instead , vary with the probability density whose shape is the only feature determining the maxima .  in the case of normally distributed data ,the maximization of cumulant function yields the first principal component for all values of : the elliptically symmetric distribution is characterized by the two tails along the major axis of the ellipse , i.e. the first principal component .when the method is applied to skew - normal and gamma distributions , for non - small , the maxima of the cumulant function determine large anomalies : high probability directions far from the center of mass .note that the limit radius is the innovative key from a technical point of view , allowing for analytical solutions . using the limit , we were also able to demonstrate that the solutions of our algorithm correspond , in general , to the directions along which the marginal probability density display the fattest tails .  using the cumulant function is computationally cheap, there is no free parameter , and has the advantage of searching for local solutions , all of which are of interest .when a solution is found , is always a good solution , in contrast with neural networks applications , for which it must be questioned if it is the global or just a local solution . in real applications( see * ? ? ?* ) , the radius must be taken as large as possible , until the expected error in the estimate of the cumulant function , due to the finite sample , reach a tolerance value ( see * ? ? ?this corresponds to maximize a combination of cumulants which is of the highest reliable order with the given amount of data , accounting for the available set of anomalies .the solutions of our algorithm are expected to transform continuously as the radius varies .hence , even if the limit can not be taken in practice , the solutions for a finite value of are expected to represent a substantial departure from the pca solution , towards the formal solution at . from the theoretical point of view , future work could be devoted to studying more in detail the nature of solutions at varying .for instance , one could attempt to find under which conditions and to what extent the solutions are in between the pca solution and the formal solution at infinite . from the applicative point of view, we expect several datasets ( e.g. * ? ? ?* ) to be fruitfully analyzed with our new method .note that the logarithm is taken for illustrative purposes : the moment generating function could be used instead of the cumulant function , since the maximization is invariant under application of a monotonous function .the present definition is however confortable in avoiding extremely large numbers .centering of data about the mean is also a practical step , related with the constraint of dealing with finite samples : if the limit could be really taken , the mean would be irrelevant .results of our procedure are corrupted if variables are standardized by a rescaling , since the relative scale of different directions is the key in detecting anomalies and comparing the size of tails .if variables are standardized , our procedure reduce to a special case of independent component analyisis ( ica , see ) , detecting independent components rather than large anomalies .results are also corrupted if we try to get an empirical estimate of the cumulant function when the underlying probability density decays less than exponentially fast . in that case , the cumulant function diverge , and the variance of the empirical estimate increases with the size of the sample ( ) , implying that the estimate is always unreliable .this work was supported by the european e2-c2 grant , the national science foundation ( grant : nsf - gmc ( atm-0327936 ) ) , by the weather and climate impact assessment science initiative at the national center for atmospheric research ( ncar ) and the anr - assimilex project .the authors would also like to credit the contributors of the r project .finally , we would like to thank isabella bordi , marcello petitta and alfonso sutera for valuable discussions .azzalini a. , capitanio a.(1999 ) .statistical applications of the multivariate skew - normal distribution , _j.roy.stat.soc._ , b61:579 - 602 .azzalini a. , dalla valle a.(1996 ) .the multivariate skew - normal distribution , _ biometrika _ , 83:715 - 726 .bernacchia a , naveau p , yiou p. and vrac m. ( 2007 ) . detecting spatial patterns with the cumulant function ii : an application to el nino , _ nonlinear process ._ , submitted .christiansen b. ( 2005 ) . the shortcomings of nonlinear principal component analysis in identifying circulation regimes , _j.climate_ , 18:4814 - 4823 .coles s. ( 2001 ) .an introduction to statistical modeling of extreme values , springer , berlin gonzalez - farias g. , dominguez - molina a. , gupta a.k .additive properties of skew - normal random vectors , _j.stat.plan.infer._ , 126:521 - 534 .hsieh w.w .nonlinear multivariate and time series analysis by neural network methods , _rev.geophys._ , 42 , rg1003 , doi : 1010.1029 / 2002rg000112233 - 243 .hyvarinen a , oja e ( 2000 ) .independent component analysis : algorithms and applications , _ neural networks _ , 13:411 - 430 kenney j.f . , keeping e.s .cumulants and the cumulant - generating function , in mathematics of statistics , princeton , nj .kotz s. , balakrishnan n. , johnson n.h .continuous multivariate distributions , john wiley and sons , new york kramer m.a .nonlinear principal component analysis using autoassociative neural networks , _j.amer.inst.chem.eng._ , 37:233 - 243 .malthouse e.c .. limitations of nonlinear pca as performed with generic neural networks , _ ieee trans.nn_ , 9 : 165 - 173 .monahan a.h . , pandolfo l. , fyfe j.c .the preferred structure of variability of the northern hemisphere atmospheric circulation , _ geophys.res.lett._ , 28:1019 - 1022 .rencher a.c .multivariate statistical inference and applications , john wiley and sons , new york sornette d. ( 2000 ) .critical phenomena in natural sciences , springer - verlag , berlin

Abstract: in climate studies , detecting spatial patterns that largely deviate from the sample mean still remains a statistical challenge . although a principal component analysis ( pca ) , or equivalently a empirical orthogonal functions ( eof ) decomposition , is often applied on this purpose , it can only provide meaningful results if the underlying multivariate distribution is gaussian . indeed , pca is based on optimizing second order moments quantities and the covariance matrix can only capture the full dependence structure for multivariate gaussian vectors . whenever the application at hand can not satisfy this normality hypothesis ( e.g. precipitation data ) , alternatives and/or improvements to pca have to be developed and studied .   to go beyond this second order statistics constraint that limits the applicability of the pca , we take advantage of the cumulant function that can produce higher order moments information . this cumulant function , well - known in the statistical literature , allows us to propose a new , simple and fast procedure to identify spatial patterns for non - gaussian data . our algorithm consists in maximizing the cumulant function . to illustrate our approach , its implementation for which explicit computations are obtained is performed on three family of of multivariate random vectors . in addition , we show that our algorithm corresponds to selecting the directions along which projected data display the largest spread over the marginal probability density tails .


graphics programs in high energy physics experiments are necessary to visualize the detectors geometry and display physics events .they help in the commissionning phase of the experiment in debugging and understanding the systems .they must provide an interactive access to the event data and their reconstruction through online and offline algorithms .as such they play an important role in the search and discovery of new physics .persint ( perspectively interacting ) is an interactive visualization program  which is developed primely for the atlas collaboration  .the display of objects and the interactivity between the user and objects or applications is realized through the use of the higz package  .facilities dedicated to the spatial navigation and the definition of the visualization properties are provided , in order to interactively set the viewing and viewed points , so as obtain the desired perspective . inparallel , applications may be launched through the use of dedicated interfaces , such as the interactive reconstruction and display of physics events .persint was originally developed as a tool for debugging and optimizing the atlas muon pattern and track reconstruction algorithm muonbox  .it was early realized that traditional event displays based on simple projections were not fit to the task of visualizing events in the toroidal , inhomogeneous field of the muon spectrometer .only a three - dimensional interactive tool with navigation capabilities is well suited for such a task .the program is now routinely used for event display of hits and reconstruction objects of complex monte carlo events and in testbeam applications , both online and offline .persint is used for the visualization of detector geometries : as such it is interfaced to amdb ( the atlas muon database ) and the agdd - xml atlas generic detector description .other applications include the visualization of the magnetic field , the level-1 muon trigger logic , the interactive generation of tracks , and the display of xml event hits .the program offers the following features :  * 3-dimensional representation of objects in full volumes or wire frames * computation of hidden faces * highlight of volumes edges * lighting intensity effects on volume facets * detection of clashing volumes and highlight of intersections * boolean volume operations ( addition , subtraction , intersection ) * spatial navigation with real - time displacements * focal length adjustable at will ( from isometry to wide - angle ) * interfaces and access to applications * save to postscript * documentation   the core of the program is written in fortran90 .dynamic data objects are extensively exploited in order to minimize the memory usage which can become important when a large numbers of volume are displayed , a situation typical to modern hep experiments , both with respect to the detector description and the event data themselves .persint uses the following features of f90 :  * usage of modules , with procedure interfaces * polymorphism * recursive functions * dynamic arrays * allocate , for local memory management * pointers * string operations * array manipulations * free source forms  the graphics interface is higz  and in general the program makes extensive use of the cernlib  .it is built on 26000 lines on code and is part of the saclay muon software suite . as such it is interfaced directly to the geometrical database amdbsimrec ( 13000 lines ) , the magnetic field database bfielddtb ( 3500 lines ) and the track reconstruction algorithm muonbox ( 58000 lines )  .the code is highly modular , organized in patches .the xml section is such an example of a patch .it contains everything needed to parse , store transiently and generates volumes from standardized xml files .the computation of volumes with hidden faces is based on the analytical computation of facets edges in space .this computation also provides the detection of clashes and the highlights of the volumes intersections , and supports boolean volume operations .the program is designed to operate in standalone mode , keeping the possibility of integration in frameworks .it was for example successfully used in the context of the online software framework of the atlas testbeams .the program is released officially at cern on afs in a public directory  since version 2.00 for three platforms : linux , sun - solaris unix and compaq / alpha unix .this repository contains the source and executable files together with documentation and useful input databases , and for developers , the tools necessary to build , compile and load the code .the latest released version is 3.00 .the documentation can be found in the persint manual located in the public repository or on the dedicated persint web page .this page provides updates on the releases .the program is designed around the command window and the higz display window . a typical view obtained with the display windowis shown in figure [ layout ] .the command window is used essentially to load files and type commands interpreted by the code and documented in the persint manual .the display window however is organized in such a way to provide maximum interactivity to avoid typing commands as much as possible .the display window is dedicated to the colour display of both the objects computed by the program for visualization and the graphical interfaces .the primary interface is the navigator .the navigator supports the navigation facilities necessary to survey the entire detector space and gives access to the various graphics applications provided by persint .its layout is shown in figure [ nav ] .it consists in an ensemble of push buttons and gauges which can be clicked , so as to modify the visualization properties or access lower level interfaces .   the spatial navigation , as well as the modification of the visualization properties ,are realized through the use of gauges which define the amplitudes of the movements of the viewing and viewed points and of the change in focal length of the observer eye , such as to obtain the desired perspective .each action is validated with the * go * " push button .alternatively , the navigation can be performed through real time displacements of the observer point .continuous displacements are obtained by dragging the cursor over the display itself , thus bypassing the use of the navigator .persint offers the possibility to modify interactively the visualization focal length . in the navigator, a sectioned gauge may be used to select the normal view , equivalent to a focal length of 35 mm , the isometric view , equivalent to an infinite focal length , or to increase or decrease the focal length at will .moreover , x , y , z , or projections of the volumes on display may be obtained by clicking dedicated push buttons ( see figure [ nav ] ) . in the case of x , y , and z projections , the observing point position ( e.g. x or x ) can be modified by clicking the displace viewing point " 2d gauge on the corresponding half . in the case of projection , this same gauge can be used to define the value of . examples of visualization of the same volumes with different perspectives are presented in figure [ expers ] .the interactive reconstruction and event display ( evd ) application of persint is dedicated to the reading of event hit files , the visualization of these hits , and the reconstruction of the event through coupling to the muonbox program  .the three - dimensional view of complex events is used to understand complex events and debug the reconstruction algorithms .figure [ evd ] shows a display of a monte carlo event in the atlas muon spectrometer in the context of a high background environment .such events are generated with the geant3-based dice simulation  .all hits and activated muon chambers are displayed .the reconstructed track is displayed in blue .visual inspection confirms the efficiency of the reconstruction algorithm in terms of pattern recognition and fitting .the projective views allows to further examine the events , though it is necessary to restrict the investigation to a limited region of the apparatus . zooming on detectorsis performed to very carefully check the event properties and reconstruction consistency , as illustrated in figure [ evddcbil ] .    in figure[ evd ] one notes the interactive reconstruction and event display interface opened on top of the navigator .this interface is equipped with an ensemble of push - buttons to set the visual properties of the displayed event and to activate the reconstruction .a separate embedded window is used to display the statistics of the event .the statistics consist essentially in the number of hits of the event , the number of hits taken into account in the candidate tracks , the reconstructed momentum , and if required from the datacards in case of physics channels analysis , the reconstructed invariant mass .the interfacing between persint and muonbox makes it possible to act interactively on the event characteristics to understand which role they play .for instance , a hit tube or a strip can be removed interactively from the event .then , when re - running the reconstruction algorithms , this removed tube will not be taken into account .the persint evd was used in the design and optimization phase of the muon spectrometer , participating in the performance studies of the apparatus and reconstruction software  .it then contributed in refining the performances of this software on complex monte carlo physics channels  .the program is used in the context of muon testbeams to display events on either online of offline mode  .event displays are useful to testbeams in order to validate the commissioning of the entire chain of detectors and data acquisition .when data is recorded in a routine mode , the evd participate in the online monitoring and the quality control of the dataflow .an example of testbeam event is shown in figure [ tb ] .it consists in a staight muon track reconstructed in the barrel stand of the h8 2002 muon system tests  .chambers misalignment is corrected using conditions alignment data .powerful graphics tool are necessary to develop and debug the geometrical descriptions of the complex hep detectors .not only 3d viewers and projective capabilities must be used , but also the possibility to automatically detect geometrical conflicts .an essential feature of geometry visualization programs is to be able to delineate precisely the volume edges .this is of primary importance in order to clearly identify the volumes boundaries in space and their possible interferences .such visual property offers a much better understanding and comprehension of the detector geometry to the end user .modern geometrical description in hep makes extensive use of boolean volume operations , which greatly simplifies the task of describing complicated geometries .visualization tools must therefore support such technology .the persint program is interfaced to the two detector description used for the atlas muon spectrometer : agdd - xml and amdb .the atlas generic detector description is based on standardized xml files .such a description is used to describe the muon spectrometer geometry  .this description is implemented in the reconstruction algorithm .the persint program benefits from an xml parser to interpret the corresponding geometrical description .the working of the interface to agdd is based on the representation of a fully clickable tree - like structure allowing selection of the desired volumes for display .an operating example is given in figure [ xml ] , showing both the xml structure and the corresponding volumes , in this case one of the atlas endcap toroid .the interface offers the representation of the volume tree in its full depth .the user can deploy or hide sub - structures by clicking volume tree pointers appearing as arrows .conveniently , geometries can be visualized in parallel as they are being developped . after a possible modification , clicking on a a dedicated push button of the navigator( see figure [ nav ] ) results in the reloading of the database .the atlas muon database is an object - oriented database for simulation and reconstruction .it is extensively used in the detectors and physics performance studies conducted since the origin of atlas  . in particular , it is used synchroneously by the dice geant3 simulation of the experiment  and the muon reconstruction program muonbox  .this database is heavily used to study the ongoing changes to the spectrometer layout  .such changes are caused by the need to take into account integration and access constraints .also , the possible staging of atlas will lead to multiple geometrical configuration , to be optimized by simulation . in this context of evolving geometries, persint provides a tool to carefully scrutinize and validate the geometrical databases .one of the important capabilities of persint is to detect and highlight intersecting or clashing volumes .an illustration is given in figure [ clash ] , where a conflict between a muon chamber and the feet of the detector is observed .persint offers the rare feature of being able to compute fully and display volumes resulting from boolean operations .being given two initial supervolumes ( a supervolume is an ensemble of volumes ) , the program is able to perform the three basic operations ( addition , subtraction , intersection ) to generate a new supervolume .this facility is independent of the complexity of the initial objects .an example is shown in figure [ labelvouss ] , which illustrates how the program can generate the voussoirs of the barrel toroid .    using this feature of persint , it is possible to generate complex - shaped volumes from very basic objects .the program provides , through the command mode , a set of instructions allowing to generate basic volumes and to perform boolean operations .the persint program is interfaced to a variety of application .they are all accessed through the navigator ( see figure [ nav ] ) .the level 1 interface is the application allowing the interactive access to the level-1 muon trigger configuration datafiles and the visual representation of the variety of objects involved in the trigger decision chain .this interface is extensively described in  .the magnetic field interface provides a fully interactive 3d visualization of the magnetic field and the magnet elements .this application is coupled to the bfielddtb set of subroutines that read the field database and compute the field at any given point .the field is visualized by means of arrows located at a series of lattice points .an example is shown in figure [ magfield ] .the direction and length of the arrow correspond to the direction and magnitude of the b - field .the lattice , either 2d or 3d , is customized by the user in terms of mesh and spatial limits , allowing to build the vector field either in very localized regions or over large distances .the muon tracks interface is used to generate and compute tracks of muons throughout the entire detector system .it is coupled to muonbox , to the detector description and the magnetic field database , in order to compute the track s trajectory , taking into account the deflections in the field , the multiple scattering , and the energy loss in matter .performances are difficult to estimate , as the visualization programs have too face many possible situations in terms of user demand , and number of volumes to display .a test is performed here as follows : the display and navigation capability is estimated on a 2.8 ghz pentium 4 .it requires 0.65 s to compute and display the 2300 voulumes with 15000 facets of the muon precision chambers system .the real - time displacements are fluid (  10 views per second ) for a number of volumes  400 .an event scan on testbeam data provides another performance measurement . in the exerciseconsisting in the display of events in loop with 6 chambers and typically 20 hits , it needs 0.02 s / event for bare evd .this an order of magnitude less than the time needed to reconstruct the event : 0.20 s / event .the present design of the program has strong advantages .the user can download a single executable binary files .no special dynamic library loading , no fancy apis need to be installed . the program is efficient even if operated remotely , e.g. on remote clusters from a x - terminal .  on the other side ,the program has shortcomings .it uses a limited graphics interface ( 256 colors in the current version of higz , possibly extended to 1024 ) .it does not profit from the high performance capabilities of graphics cards when operated locally on pcs ( z - buffer ) .  in the developments under consideration , a migration to opengl is envisaged . however some of the features of the present design would be lost .for example , such technology based on z - buffer does nt allow to delineate the volume edges .this property is crucial for hep detectors .it is considered to integrate parts of persint in the root data analysis framework .the graphics capabilities of root would be enhanced by using the calculus of hidden faces and intersecting volumes . in parallel , the integration of persint in the athena atlas software framework for reconstruction and analysis can be studied .the utilization of persint will continue as an event and geometry display for the atlas experiment .it will be used as online evd in the h8 - 2003 muon testbeam , and in the h8 - 2004 atlas combined testbeam .it will have an important role to play in the commissioning of atlas , participating in the debug of the detector ( cosmics run , calibration runs , ... ) .finally it is a candidate for the atlas online evd for monitoring of data - taking .higz high level interface to graphics and zebra user s guide , cern program library long writeups * q120 * , application software group , computing and networks division , cern , geneva , switzerland , june 1995 .

Abstract: the persint program is designed for the three - dimensional representation of objects and for the interfacing and access to a variety of independent applications , in a fully interactive way . facilities are provided for the spatial navigation and the definition of the visualization properties , in order to interactively set the viewing and viewed points , and to obtain the desired perspective . in parallel , applications may be launched through the use of dedicated interfaces , such as the interactive reconstruction and display of physics events . recent developments have focalized on the interfacing to the xml atlas general detector description agdd , making it a widely used tool for xml developers . the graphics capabilities of this program were exploited in the context of the atlas 2002 muon testbeam where it was used as an online event display , integrated in the online software framework and participating in the commissioning and debug of the detector system .


discrete graphical markov models are models for discrete distributions representable by graphs , associating nodes with the variables and using rules that translate properties of the graph into conditional independence statements between variables .there are several classes of graphical models ; see for a review . in this paperwe focus on the class of multivariate regression chain graphs and we discuss their definition and parameterization for discrete variables .multivariate regression chain graphs generalize directed acyclic graphs , which model recursive sequences of univariate responses , by allowing multiple responses . as in all chain graph modelsthe variables can be arranged in a sequence of groups , called chain components , ordered on the basis of subject - matter considerations , and the variables within a group are considered to be on an equal standing as responses .the edges are undirected within the chain components , drawn as dashed lines , , or as bi - directed arrows , , and directed between components , all pointing in the same direction , i.e. , with no chance of semi - directed cycles .one special feature of multivariate regression chain graphs is that the responses are potentially depending on all the variables in all previous groups , but not on the other responses .chain graphs with this interpretation were proposed first by cox and wermuth in , with several examples in ( * ? ? ?* chapter 5 ) .in the special case of a single group of responses with no explanatory variables , multivariate regression chain graphs reduce to covariance graphs i.e. , to undirected graphs representing marginal independencies with the basic rule that if any its subgraph is disconnected , i.e. , composed by completely separated sets of nodes , then the associated variables are jointly independent ; see and . in the general case ,the interpretation of the undirected graphs within a chain component is that of a covariance graph , but conditional on all variables in preceding components .for example , the missing edge in the graph of figure [ fig : cg](b ) is interpreted as the independence statement , compactly written in terms of nodes as .  the interpretation of the directed edges is that of multivariate regression models , with a missing edge denoting a conditional independence of the response on a variable given all the remaining potential explanatory variables .thus , in the chain graph of figure [ fig : cg](a ) the missing arrow indicates the independence statement .the interpretation differs from that of classical chain graphs , ( lauritzen , wermuth , frydenberg , , , lwf for short ) where the missing edges mean conditional independencies given all the remaining variables including the current responses .however , in studies involving longitudinal data , such as the prospective study of child development discussed in where there are blocks of joint responses recorded at ages of 3 months , 2 years and 4 years , an analysis conditioning exclusively on the previous developmental states is typically appropriate .recently , distinguished four types of chain graphs comprising the classical and the alternative ( andersson , madigan , perlman , ) chain graph models , called of type i and ii , respectively . in this paperwe give a formal definition of multivariate regression chain graph models and we prove that they are equivalent to the chain graph models of type iv , in drton s classification .moreover , we provide a parameterization based on recursive multivariate logistic regression models .these models , introduced in ( * ? ? ?* section 6.5.4 ) and can be used to define all the independence constraints .the models can be defined by an intuitive rule , see theorem [ prop : base ] , based on the structure of the chain graph , that can be translated into a sequence of explicit regression models .one consequence of the given results is that any discrete multivariate regression chain graph model is a curved exponential family , a result obtained in with a different proof .the basic definitions and notation used in this paper closely follow , and they are briefly recalled below .a chain graph is a graph with finite node set and an edge set that may contain either directed edges or undirected edges .the graph has no semi - directed cycle , i.e. , no path from a node to itself with at least one directed edge such that all directed edges have the same orientation .  ; : ; : .dashed lines only occur within chain components ., width=415 ]  the node set of a chain graph can be partitioned into disjoint subsets called _chain components _ , such that all edges in each subgraph are undirected and the edges between different subsets are directed , pointing in the same direction . for chain graphs with the multivariate regression interpretation the subgraphs within each chain component have undirected dashed ( ) or bi - directed ( ) edges .the former convention is adopted in this paper .thus , the chain graph of figure [ fig : cg](c ) has three chain components , while the previous ones have two components .  given a subset of nodes within a chain component , the subgraph , is said to be _ disconnected _ if there exist two nodes in such that no path in has those nodes as endpoints .in this case , can be partitioned uniquely into a set of connected components .otherwise , the subgraph is _connected_. for example , in chain graph ( c ) of figure [ fig : cg ] , the subgraph with is disconnected with two connected components and . on the other hand the subgraph with connected . in the remainder of the paper, we shall say for short that a subset of nodes in a component is connected ( resp .disconnected ) if the subgraph is connected ( resp .disconnected ) .any chain graph yields a directed acyclic graph of its chain components having as node set and an edge whenever there exists in the chain graph at least one edge connecting a node in with node in . in this directed graph , we may define for each the set as the union of all the chain components that are parents of in the directed graph .this concept is distinct from the usual notion of the _ parents _ of a set of nodes in the chain graph , that is the set of all the nodes outside such that with .  .in ( b ) the set of predecessors of is , while the set of parent components of is . , width=377 ]  for instance , in the graph of figure [ fig : ord](a ) , for , the set of parent components is , whereas the set of parents of is .  in this paperwe start the analysis from a given chain graph with an associated collection of chain components .however in applied work , where variables are linked to nodes by the correspondence for , usually a set of chain components is assumed known from previous studies of substantive theories or from the temporal ordering of the variables . for variables within such chain components no direction of influenceis specified and they are considered as joint responses , i.e. to be on equal standing .the relations between variables in different chain components are directional and are typically based on a preliminary distinction of responses , intermediate responses and purely explanatory factors .often , a full ordering of the components is assumed based on time order or on a subject - matter working hypothesis ; see .  given a chain graph with chain components , we can always define a strict total order of the chain components that is _ consistent _ with the partial order induced by the chain graph , i.e. , such that if then .for instance , in the chain graph of figure [ fig : ord](a ) there are four chain components ordered in graph ( b ) as .note that the chosen total order of the chain components is in general not unique and that another consistent ordering could be .in the remainder of the paper we shall assume that a consistent ordering of the chain components is given .then , for each , the set of all components preceding is known and we may define the cumulative set of nodes contained in the predecessors of component that we sometimes also call the past of .the set captures the notion of all the potential explanatory variables of the response variables within . by definition ,as the full ordering of the components is consistent with , the set of predecessors of each chain component always includes the parent components .the following definition explains the meaning of the multivariate regression interpretation of a chain graph .[ def:1 ] let be a chain graph with chain components and let define an ordering of the chain components consistent with the graph .a joint distribution p of the random vector obeys the ( global ) multivariate regression markov property for if it satisfies the following independencies . for all and for all : \mid \operatorname{pa}_g(a ) ,  \\( \textsc{mr2})\quad  & \text { if is  disconnected with connected components }  a_1 , \dots , a_r:\\  & a_1 { \mbox{\protect } } \cdots { \mbox{\protect } } a_r  \mid \operatorname{pre}(t ) .\end{aligned}\ ] ]  assuming that the distribution has a density with respect to a product measure , the definition can be stated by the following two equivalent conditions :    for all and for all connected subset .  for all and for all disconnected subset with connected components .  in words , for any connected subset of responses in a component , its conditional distribution given the variables in the past depends only on the parents of .on the other hand , if is disconnected ( i.e. , the subgraph is disconnected ) the variables in its connected components , are jointly independent given the variables in the past .definition [ def:1 ] gives a local markov property which always implies the following pairwise markov property : for every uncoupled pair of nodes then in particular , two pairwise independencies and can occur only in combination with the joint independence .this means that in the associated model the composition property is always satisfied ; see .thus , even though we concentrate in this paper on the family of multinomial distributions which does not satisfy the composition property , the models in which mr1 and mr2 hold have this property .one immediate consequence of definition [ def:1 ] is that if the probability density is strictly positive , then it factorizes according to the directed acyclic graph of the chain components : this factorization property is shared by all types of chain graphs ; see and .recently , discussed four different block - recursive markov properties for chain graphs , of which we discuss here those with the _markov property of type iv_. to state it we need two further concepts from graph theory . given a chain graph ,the set is the union of itself and the set of nodes that are _ neighbours _ of , i.e. coupled by an undirected edge to some node node in .moreover , the set of _ non - descendants _ of a chain component , is the union of all components such that there is no directed path from to in the directed graph of chain components .( chain graph markov property of type iv ) ; .[def : drton ] let be a chain graph with chain components and directed acyclic graph of components .the joint probability distribution of obeys the block - recursive markov property of type iv if it satisfies the following independencies : \mid \operatorname{pa}_d(t ) \text { for all } t \in { \mathcal{t}}\\ ( \textsc{iv1})\quad &  a { \mbox{\protect } } [ \operatorname{pa}_d(t ) \setminus \operatorname{pa}_g(a ) ] \mid \operatorname{pa}_g(a ) \text { for all } t\in { \mathcal{t } } , \text { for all } a \subseteq t\\ ( \textsc{iv2 } ) \quad &  a { \mbox{\protect } } [ t \setminus \mathrm{nb}_g(a ) ] \mid \operatorname{pa}_d(t ) \text { for all } t \in { \mathcal{t}}\text { for all  connected subsets }  a \subseteq t. \end{aligned}\ ] ]  then , we have the following result , proved in the appendix .  [ theo:1 ]given a chain graph , the multivariate regression markov property is equivalent to the block - recursive markov property of type iv .this result shows that the block - recursive property of a chain graph of type iv is in fact simplified by definition [ def:1 ] . on the other hand definition [ def:1 ]depends only apparently on the chosen full ordering of the chain components , because the equivalent definition [ def : drton ] depends only on the underlying chain graph .the independencies implied by the multivariate regression chain graph markov property are illustrated below for each of the graphs of figure [ fig : cg ] .graph ( a ) represents the independencies of the seemingly unrelated regression model ; see , . for and have the independencies and .note that for the connected set the condition ( mr1 ) implies the trivial statement .  in graph( b ) one has and .thus , for each connected subset , by ( mr1 ) , we have the non - trivial statements then , for the remaining disconnected set we obtain by ( mr2 ) the independence .  in graph ( c ) , considering the conditional distribution for and we can define independencies for each of the eight connected subsets of . for instancewe have the last independence is equivalent to the factorization of the joint probability distribution according to the directed acyclic graph of the chain components .the remaining five disconnected subsets of imply the conditional independencies and notice that when in a component there are two uncoupled nodes , then there is a conditional independence given simply the common parents of the two nodes .for example , in graph ( c ) , we have not only but also .[ rem : dec ] when each component induces a complete subgraph and , for all subsets in , the set of parents of , , coincides with the set of the parent components of , , then the only conditional independence implied by the multivariate regression markov property is | \operatorname{pa}_d(t ) , \text { for all } a \subseteq t ,  \quad t \in { \mathcal{t}}.\ ] ] this condition is in turn equivalent just to the factorization of the joint probability distribution .  in definition [ def:1 ] , ( mr2 )is equivalent to imposing that for all the conditional distribution , satisfies the independencies of a covariance graph model with respect to the subgraph .  in (* proposition 3 ) , it is shown that a covariance graph model is defined by constraining to zero , in the multivariate logistic parameterization , the parameters corresponding to all disconnected subsets of the graph . in the following subsectionwe extend this approach to the multivariate regression chain graph models .let be a discrete random vector , where each variable has a finite number of levels .thus takes values in the set whose elements are the cells of the joint contingency table , denoted by . the first level of each variableis considered a reference level and we consider also the set of cells having all indices different from the first .the elements of are denoted by .the joint probability distribution of is defined by the mass function or equivalently by the probability vector . with three variables we shall use often instead of .  given two disjoint subsets and of , the marginal probability distribution of is where is a subvector of belonging to the marginal contingency table .the conditional probability distributions are defined as usual and denoted by , for and or , compactly , by .when appropriate , we define the set .  a discrete multivariate regression chain graph model associated with the chain graph is the set of strictly positive joint probability distributions , for that obeys the multivariate regression markov property . by theorem [ theo:1 ]this class coincides with the set of discrete chain graph models of type iv .  in the next subsectionwe define an appropriate parameterization for each component of the standard factorization of the joint probability distribution .actually we define a saturated linear model for a suitable transformation of the parameters of each conditional probability distribution .the suggested link function is the multivariate logistic transformation ; see , .this link transforms the joint probability vector of the responses into a vector of logistic contrasts defined for all the marginal distributions .the contrasts of interest are all sets of univariate , bivariate , and higher - order contrasts . in general , a _ multivariate logistic contrast _ for a marginal table is defined by the function where the notation denotes the cardinality of set .the contrasts for a margin are denoted by and the full vector of the contrasts for all nonempty margins are denoted by .the following example illustrates the transformation for two responses .[ ex : mlogit ] let , for , be a joint bivariate distribution for two discrete variables and .then , the multivariate logistic transform changes the vector of probabilities , belonging to the 5-dimensional simplex , into the vector where the suffix indicates summing over the corresponding index .thus , the parameters and are marginal baseline logits for the variables and , while is a vector of log odds - ratios .the definition used in this paper uses baseline coding , i.e. , the contrasts are defined with respect to a reference level , by convention the first .therefore the dimension of the vectors , and are the dimensions of the sets , and .other coding schemes can be adopted , as discussed for instance in and .[ rem : diff ] this transformation for multivariate binary variables is discussed in , where it is shown that the function from to is a smooth ( ) one - to - one function having a smooth inverse , i.e. it is a diffeomorphism ; see also . for general discrete variablessee .the parameters are not variation - independent , i.e. they do not belong to a hyper - rectangle .however , they satisfy the _upward compatibility property _, i.e. , they have the same meaning across different marginal distributions ; see and ( * ? ? ?* proposition 4 ) . often the multivariate logistic link is written as where and are suitable kronecker products of contrast and marginalization matrices respectively . for the explicit construction of these matrices see .we specify the dependence of the responses in each component on the variables in the past by defining a saturated multivariate logistic model for the conditional probability distribution . the full saturated model for the joint probability then follows from the factorization .  for each each covariate class  be the vector with strictly positive components for .then consider the associated conditional multivariate logistic parameters defined using the link function .notice that this vector is composed of contrasts for all nonempty subsets of .then we express the dependence of each of them on the variables in the preceding components by a complete factorial model here the vectors have dimensions of the sets , and are defined according to the baseline coding , and thus vanish when at least one component of takes on the first level . again, here different codings may be used if appropriate .often is useful to express in matrix form where is the column vector obtained by stacking all vectors for , is a full - rank design matrix and is a parameter vector .suppose that in example [ ex : mlogit ] the responses and depend on two binary explanatory variables and , with levels indexed by and , respectively .then , the saturated model is for .the explicit form of the matrix in equation is , using the kronecker product operator , i.e. , a matrix of a complete factorial design matrix , where is an identity matrix of order equal to the common dimension of each . following we shall denote the model , for the sake of brevity , by a multivariate model formula where is the factorial expansion in wilkinson and rogers symbolic notation ; .  when we need to express the overall 1 - 1 smooth transformation of the conditional probability vectors , denoted collectively by , into the logistic and regression parameters we introduce the vectors and obtained by concatenating the parameters and , respectively , for all nonempty subsets of , writing where is a full rank block - diagonal matrix of the saturated model , and where and are suitable overall contrast and marginalization matrices .a multivariate regression chain graph model is specified by zero constraints on the parameters of the saturated model .we give first an example and then we state the general result .  continuing the previous example for the chain graph of figure [ fig : cg](a ), we shall require that depends only on and depends only on .therefore , we specify the model with a corresponding multivariate model formula the reduced model satisfies the two independencies and because the first two equations are equivalent to and , respectively .the log odds - ratio between and , on the other hand , depends in general on all the combinations of levels of the two explanatory variables .  the following theorem , proved in the appendix states a general rule to parametrize any discrete chain graph model of multivariate regression type .  [ prop : base ]let be a chain graph and let be a consistent ordering of the chain components .a joint distribution of the discrete random vector belongs to if and only if , in the multivariate logistic model , the parameters , , whenever    for all and for all .notice that equations and correspond to conditions and respectively of definition [ def:1 ] .thus the multivariate regression chain graph model turns out to be if is connected and if is disconnected . in matrix form we have a linear predictor where is the matrix of the reduced model obtained by removing selected columns of , and are the associated parameters .  the proof of theorem [ prop : base ] is based on a basic property of the regression parameters of model , i.e. , that they are identical to log - linear parameters defined in selected marginal tables . specifically , each coincides with the vector of log - linear parameters of order in the marginal table .see lemma [ prop : loglin ] in the appendix .theorem [ prop : base ] shows also that the chain graph model is defined by a set of linear restrictions on a multivariate logistic parameterization and thus is a curved exponential family .[ ex:3 ] from theorem [ prop : base ] , the chain graph model of figure [ fig : cg](b ) is defined by the equations and by the multivariate logistic model formula   .marginal log - linear parameters of the saturated model for a discrete multivariate logistic model with three responses and two explanatory variables .each row lists log - linear parameters defined within a marginal table indicated in the last column . in boldface the nonzero terms of the chain graph model of example [ ex:3 ] .the shaded part of the table collects the interactions of order higher than two .[ cols= " > , > , > , > , > , > " , ]   notice that the marginal logit of does not depend on the variables .this is denoted by . on the other hand ,the missing edge with associated independence , implies that the bivariate logit between and is zero , denoted by model formula .the above equations reflect exactly the independence structure encoded by the multivariate regression markov property but leave a complete free model for the three variable logistic parameter .table [ tab:1 ] lists the parameters ( and their log - linear interpretations ) of the saturated model .the nonvanishing parameters of the chain graph model are in boldface .the shaded portion of the table indicates the interactions of order higher than two .therefore , the chain graph model contains 6 parameters in the shaded area that have a more complex interpretation and that are not strictly needed to define the independence structure .this suggests to consider as a starting model a multivariate logistic regression model with no parameters of log - linear order higher than two and then using a backward selection strategy to test for the independencies .some adjustment of the procedure is needed to include selected higher - order interactions when needed .notice also that the parameters in table [ tab:1 ] form a marginal log - linear parameterization in the sense of bergsma and rudas , , a result that can be proved for any discrete multivariate regression chain model .a parallel multivariate logistic parameterization for the model can be obtained from definition [ def : drton ] and the associated characterization in terms of densities of lemma [ prop : iv ] , in the appendix . in this case , using the factorization , the multivariate logistic models can be defined in the lower - dimensional conditional distributions .therefore we state the following corollary .the joint probability distribution of the random vector belongs to if and only if it factorizes according to equation , and for each conditional distribution , for , the multivariate logistic parameters are   the class of models defined in remark [ rem : dec ] , corresponding exactly to the factorization , all the independencies are obtained by setting for all in equation .   the estimation of discrete multivariate regression chain models can be carried out by fitting separate multivariate logistic regression models to each factor of the decomposition . specifically , given a block of responses , and the group of covariates we consider the table of frequencies for each covariate class , where is an index numbering the cells of the marginal table .then we assume that each is multinomial with .given independent observations the vector has a product - multinomial distribution and the log - likelihood is where , and , from .the maximization of this likelihood under the above linear constraints has been discussed by several authors ; see , , , , among others .we give a simple illustration based on an application to data from the u.s .general social survey , for years 1972 - 2006 .the data are collected on 13067 individuals on 5 variables .there are three binary responses concerning individual opinions ( 1 = favor , 2 = oppose ) on legal abortion if pregnant as a result of rape , , on death penalty for convicted of murder , , and on the introduction of police permits for buying guns , .the potentially explanatory variables considered are , job satisfaction ( with three levels 1 = very satisfied , 2 = moderately satisfied , 3 = a little or very dissatisfied ) and , gender ( 1=male , 2=female ) .we can interpret responses and as indicators of the attitude towards individual safety , while and as indicators of the concern for the value of human life , even in extreme situations .the two explanatory variables turned out to be independent ( with a likelihood - ratio test statistic of , 1 d.f . ) .hence , we concentrate on the model model for the conditional distribution . herethe saturated model has 42 parameters and the structure of the parameters is that of table [ tab:1 ] , with the only modification of the dimensions of the interaction parameters involving the factor , with three levels .  lrrclrrrr model for & deviance & d.f .& & logit & const . & & & + ( 1 ) and & 12.84 & 10 & & & 0.766 & & & 0.766 + ( 2 ) no 5-factor interaction & 0.49 & 2 & & & & & & + ( 3 ) + no 4-factor interactions & 5.59 & 11 & & & & & & + ( 4 ) + no 3-factor interactions & 30.16 & 27 & & & & & & + ( 6 ) + delete edge & 33.38 & 28 & & & & & & + ( 7 ) + delete edge & 34.25 & 30 & & & & & & +  we describe a hierarchical backward selection strategy . for this , we examine first the sequence of models obtained by successively removing the higher - order interactions , see table [ tab:1 ] .then , we drop some of the remaining terms to fit independencies .the results are shown in table [ tab:2 ] . the model with no interactions of order higherthen three has a deviance of with 27 degrees of freedom adequate . from the edge exclusion deviances ,we verify that we can remove the edges ( , 1 d.f . ) , and ( , 2 d.f . )the final multivariate regression chain graph model is shown in fig .[ fig:3](a ) has a combined deviance of on 32 degrees of freedom .notice that the model includes independence and non - independence constraints , the latter following for our preference for model with all interpretable parameters .the chain graph model corresponding exactly to the implied independencies has far more parameters , with a deviance of against 12 degrees of freedom . while this model is adequate the chosen model has a simpler interpretation .the fitted parameters are shown in table [ tab:2 ] right .the first three rows give the parameters of three univariate logit regressions for being in favor of the issue . measure the effect of moderate and full job satisfaction , respectively , with respect to a baseline level of no statisfaction and is the effect of females .= ) .the final fitted model including further non - independence constraints has a deviance = on d.f .( b ) the best fitting lwf chain graph model ( deviance = , d.f .= ).,width=302 ]  thus the effect of increased job satisfaction whatever the gender , is to increase the probability of being in favor of capital punishment , and against abortion .women , are more favorable than males to gun s regulation and against death penalty and abortion , other thing being equal .the negative residual association between and and the positive one between and having accounted for gender and job satisfaction , are as expected . as a comparison , in this example , a best fitting classical chain graph model with lwf interpretation , has one additional edge , as shown in figure [ fig:3 ] .the multivariate regression chain graph has a simpler interpretation in terms of three additive logistic regressions and two residual associations interpretable as deriving from two latent variables .we thank nanny wermuth , d. r. cox and two referees for helpful comments .the work was supported by miur , rome , under the project prin 2007 , xecz7l001/2 .24 natexlab#1#1url # 1`#1`urlprefix[2][]#2  andersson , s. , madigan , d. and perlman , m. ( 2001 ) .alternative markov properties for chain graphs .. j. statist ._ , * 28 * 3385 .  ,colombi , r. and forcina , a. ( 2007 ) . an extended class of marginal link functions for modelling contingency tables by equality and inequality constraints . _ statist . sinica _ ,* 17 * 691711 .  and rudas , t. ( 2002 ) .marginal models for categorical data ._ , * 30 * 140159 .  , laucht , m. , coneus , k. and pfeiffer , f. ( 2009 ) . initial risk matrix , home resources , ability development , and children s achievment ._ j. european economic ass ._ , * 7*(2 - 3 ) , 638648 .  , dawid , a. p. , lauritzen , s. l. and spiegelhalter , d. j. ( 1999 ). _ probabilistic networks and expert systems_. springer series , new york .  and wermuth , n. ( 1993 ) .linear dependencies represented by chain graphs ( with discussion ) . _ statist ._ , * 8 * 204218 , 247277 .  and wermuth , n. ( 1996 ) ._ multivariate dependencies_. chapman and hall , london .  and marsden , j. a. ( 2007 ) ._ general social surveys cumulative codebook : 1972 - 2006_. : chicago .  ( 2009 ). discrete chain graph models ._ bernoulli _ , * 15*(3 ) 736753 .  and perlman , m. d. ( 2008 ) . a sinful approach to gaussian graphical model selection ._ j. statist .plann . inference _ , * 138*(4 ) 11791200 .  andrichardson , t. s. ( 2004 ) .multimodality of the likelihood in the bivariate seemingly unrelated regression models ._ biometrika _ , * 91 * 383392 .  and richardson , t. s. ( 2008 ) . binary models for marginal independence ._ j. r. stat ._ , * 70 * 287309 .the chain graph markov property .j. statist ._ , * 17 * 333353 .  and mccullagh , p. ( 1995 ) .multivariate logistic models ._ j. r. stat ._ , * 57 * 533546 .  ( 1995 ) . a note on multivariate logistic models for contingency tables ._ austral .j. statist ._ , * 39*(3 ) 261276 .maximum likelihood methods for a generalized class of log - linear models ._ , * 24*(2 ) 726752 ._ graphical models_. oxford university press , oxford .  and wermuth , n. ( 1989 ) .graphical models for associations between variables , some of which are qualitative and some quantitative ._ , * 17 * 3157 .  ,marchetti , g. m. and bergsma , w. p. ( 2009 ) .parameterizations and fitting of bi - directed graph models to categorical data .j. statist ._ , * 36*(3 ) 559576 .  and lupparelli , m. ( 2008 ) .parameterization and fitting of a class of discrete graphical models . in _compstat 2008 - proceedings in computational statistics : 18th symposium _ ( p. brito , ed . ) .physica - verlag , heidelberg , 117128 .  andnelder , j. a. ( 1989 ) ._ generalized linear models_. chapman and hall , london .arkov property for acyclic directed mixed graphs .j. statist ._ , * 30 * 145157 ._ probabilistic conditional independence structures_. springer series in information science and statistics , london .  and cox , d. r. ( 1992 ) . on the relation between interactions obtained with alternative codings of discrete variables ._ methodika _ , * vi * 7685 .  and cox , d. r. ( 2004 ) .joint response graphs and separation induced by triangular systems ._ j. r. stat ._ , * 66 * 687717 .  and rogers , c. e. ( 1973 ) .symbolic description of factorial models for analysis of variance ., * 22 * 392399 .we shall assume for the joint distribution the existence of a density with respect to a product measure .proofs using only basic properties of conditional independence can also be given , but are omitted for brevity .condition ( iv0 ) states that the joint probability distribution obeys the _ local directed markov property _ relative to the directed graph of the chain components .then , using the equivalence of the local and well - ordered local markov property in directed graphs applied to the graph of the components as discussed in ( * ? ? ?* appendix a ) , ( iv0 ) turns out to be equivalent to for any ordering of the components consistent with the chain graph .moreover , condition ( iv2 ) has been proved by to be equivalent to the joint independence .statement ( iv1 ) implies but in fact it can be restricted to connected subsets because for disconnected subsets it follows from and from restricted to connected sets . if is disconnected implies by applying to the connected sets and noting that .therefore , and equation ( iv1 ) follows .( definition [ def:1 ] implies definition [ def : drton ] . )equation implies for all connected because . thus implies and for , because any is connected , by definition .thus , if is disconnected gives and follows .( definition [ def : drton ] implies definition [ def:1 ] . )statement implies for , that .thus for all connected implies , i.e. , .moreover if is disconnected implies i.e. , .  given a subvector of the given random vector , the log - linear expansion of its marginal probability distribution is where defines the ` interaction ' parameters of order , in the baseline parameterization , i.e. with the implicit constraint that the function returns zero whenever at least one of the indices in takes the first level .[ prop : loglin ] if is the multivariate logistic contrast of the conditional probability distribution for subset of , then , with , where are the log - linear interaction parameters of order in the marginal probability distribution .first note that the multivariate logistic contrasts can be written then , we express the logarithm of the joint probabilities as sum of log - linear interactions using therefore , by substitution into equation we get where the last equality is obtained using mbius inversion , see ( * ? ? ?* , lemma a.2 ) ) .if holds for any chain component , then for any connected set , is a function of only .therefore , using the diffeomorphism and the property of upward compatibility discussed in remark [ rem : diff ] , the conditional distribution coincide with and condition ( mr1 ) holds .conversely , if condition ( mr1 ) holds and , for all connected subsets of , then the components of are by lemma [ prop : loglin ] , and thus holds with where denotes the vector of log - linear parameters for all .condition ( mr2 ) of definition [ def:1 ] is equivalent to imposing that , for any chain component , the conditional distribution satisfies the independence model of a covariance sub - graph . in and proved that , given a joint distribution , a covariance graph model is satisfied if and only if in the multivariate logistic parameterization , for all disconnected sets .therefore , extending this result to the conditional distribution and considering the diffeomorphism , condition ( mr2 ) holds if and only if for every disconnected set . following the factorial model ( 7 ) , with , for each disconnected subset of . notice that , by lemma [ prop : loglin ] , , with

Abstract: we discuss a class of chain graph models for categorical variables defined by what we call a multivariate regression chain graph markov property . first , the set of local independencies of these models is shown to be markov equivalent to those of a chain graph model recently defined in the literature . next we provide a parameterization based on a sequence of generalized linear models with a multivariate logistic link function that captures all independence constraints in any chain graph model of this kind .