A new successive encoding scheme is proposed to effectively generate a random vector with prescribed joint density that induces a latent Gaussian tree structure. We prove the accuracy of such encoding scheme in terms of vanishing total variation distance between the synthesized and desired statistics. The encoding algorithm relies on the learned structure of tree to use minimal number of common random variables to synthesize the desired density, with compact modeling complexity. We characterize the achievable rate region for the rate tuples of multi-layer latent Gaussian tree, through which the number of bits needed to simulate such Gaussian joint density are determined. The random sources used in our algorithm are the latent variables at the top layer of tree along with Bernoulli sign inputs, which capture the correlation signs between the variables. In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. Given the derived achievable rate region for synthesis of latent Gaussian trees, we also quantify the amount of information loss due to unrecoverable sign information. It is shown that maximizing the achievable rate-region is equivalent to finding the worst case density for Bernoulli sign inputs where maximum amount of sign information is lost.
There are several works that extend the classical bi-variate synthesis problem in Wyner's study to more general scenarios. In @cite_3 @cite_19 @cite_16 , the authors aim to define the common information of @math dependent random variables, to further address the same question in this setting. A lower bound on such generalized common information is obtained in @cite_4 . Also, the common information for a special case with @math Gaussian variables with homogeneous pairwise correlations is obtained. They resort to the same scenario as Wyner @cite_0 did, i.e., considering one random variable to define such common randomness. Veld and Gastpar @cite_9 characterize such quantity for a more general set of Gaussian vectors with circulant covariance matrices. Also, in @cite_5 the authors completely characterize the common information between two jointly Gaussian vectors, as a function of certain singular values that are related to both joint and marginal covariance matrices of two Gaussian random vectors. However, they still divide the random vector into two groups, which makes it similar to Wyner's scenario.
@cite_4 @cite_9 @cite_3 @cite_0 @cite_19 @cite_5 @cite_16
1662534456 2342676922 2553864996 2141432745 2043976992 1901718207 2244961551
We prove that by imposing a conditional mutual independence constraint and a marginalisation constraint, the almost entropic region can be completely characterised by Shannon-type information inequalities. Such a property is applied to obtain an explicit lower bound on the generalised Wyner common information. We study a caching problem that resembles a lossy Gray-Wyner network: A source produces vector samples from a Gaussian distribution, but the user is interested in the samples of only one component. The encoder first sends a cache message without any knowledge of the user's preference. Upon learning her request, a second message is provided in the update phase so as to attain the desired fidelity on that component. Wyner defined the notion of common information of two discrete random variables as the minimum of I(W; X, Y) where W induces conditional independence between X and Y. Its generalization to multiple dependent random variables revealed a surprising monotone property in the number of variables. Motivated by this monotonicity property, this paper explores the application of Wyner's common information to inference problems and its connection with other performance metrics. A central question is that under what conditions Wyner's common information captures the entire information contained in the observations about the inference object under a simple Bayesian model. For infinitely exchangeable random variables, it is shown using the de Finetti-Hewitt-Savage theorem that the common information is asymptotically equal to the information of the inference object. For finite exchangeable random variables, such conclusion is no longer true even for infinitely extendable sequences. However, for some special cases, including both the binary and the Gaussian cases, concrete connection between common information and inference performance metrics can be established even for finite samples. The problem of finding a meaningful measure of the "common information" or "common randomness' of two discrete dependent random variables X,Y is studied. The quantity C(X; Y) is defined as the minimum possible value of I(X, Y; W) where the minimum is taken over all distributions defining an auxiliary random variable W W , a finite set, such that X, Y are conditionally independent given W . The main result of the paper is contained in two theorems which show that C(X; Y) is i) the minimum R_0 such that a sequence of independent copies of (X,Y) can be efficiently encoded into three binary streams W_0, W_1,W_2 with rates R_0,R_1,R_2 , respectively, [ R_i = H(X, Y)] and X recovered from (W_0, W_1) , and Y recovered from (W_0, W_2) , i.e., W_0 is the common stream; ii) the minimum binary rate R of the common input to independent processors that generate an approximation to X,Y . We study secure source-coding with causal disclosure, under the Gaussian distribution. The optimality of Gaussian auxiliary random variables is shown in various scenarios. We explicitly characterize the tradeoff between the rates of communication and secret key. This tradeoff is the result of a mutual information optimization under Markov constraints. As a corollary, we deduce a general formula for Wyner's Common Information in the Gaussian setting. Wyner’s common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray–Wyner network, it is established that Wyner’s common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner’s common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray–Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner’s common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray–Wyner lossy source coding problem and for identifying conditions under which Wyner’s common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained.