The posterior matching scheme, for feedback encoding of a message point lying on the unit interval over memoryless channels, maximizes mutual information for an arbitrary number of channel uses. However, it in general does not always achieve any positive rate; so far, elaborate analyses have been required to show that it achieves any positive rate below capacity. More recent efforts have introduced a random "dither" shared by the encoder and decoder to the problem formulation, to simplify analyses and guarantee that the randomized scheme achieves any rate below capacity. Motivated by applications (e.g. human-computer interfaces) where (a) common randomness shared by the encoder and decoder may not be feasible and (b) the message point lies in a higher dimensional space, we focus here on the original formulation without common randomness, and use optimal transport theory to generalize the scheme for a message point in a higher dimensional space. By defining a stricter, almost sure, notion of message decoding, we use classical probabilistic techniques (e.g. change of measure and martingale convergence) to establish succinct necessary and sufficient conditions on when the message point can be recovered from infinite observations: Birkhoff ergodicity of a random process sequentially generated by the encoder. We also show a surprising "all or nothing" result: the same ergodicity condition is necessary and sufficient to achieve any rate below capacity. We provide applications of this message point framework in human-computer interfaces and multi-antenna communications.
Recently, we have considered the case when the message point lies in a higher dimensional space (e.g. @math for some arbitrary @math ) and constructed a feedback encoding scheme using optimal transport theory @cite_19 . Inspired by the time-invariant dynamical systems structure of the original PM scheme (without shared randomness at the encoder), and motivated by communication applications where humans play a role, below we develop appropriate notions of reliability and rate achievability in the multidimensional message point setting involving almost sure convergence. This allows us to use classical probability tools, including change of measure and martingale convergence, that have recently been employed by Van Handel to provide rigorous and succinct results pertaining to filter stability in hidden Markov models @cite_31 @cite_33 , to provide succinct necessary and sufficient conditions for the generalized PM scheme to attain optimal performance. This provides a clear characterization of optimality of the original PM scheme in arbitrary dimensions in terms of Birkhoff ergodicity of an appropriate random process, without requiring the use of a dither.
@cite_19 @cite_31 @cite_33
2074430484 2070178449 2010995253
This paper re-visits Shayevitz & Feder's recent ‘Posterior Matching Scheme’, an explicit, dynamical system encoder for communication with feedback that treats the message as a point on the [0,1] line and achieves capacity on memo-ryless channels. It has two key properties that ensure that it maximizes mutual information at each step: (a) the encoder sequentially hands the decoder what is missing; and (b) the next input has the desired statistics. Motivated by brain-machine interface applications and multi-antenna communications, we consider developing dynamical system feedback encoders for scenarios when the message point lies in higher dimensions. We develop a necessary and sufficient condition — the Jacobian equation — for any dynamical system encoder that maximizes mutual information. In general, there are many solutions to this equation. We connect this to the Monge-Kantorovich Optimal Transportation Problem, which provides a framework to identify a unique solution suiting a specific purpose. We provide two examplar capacity-achieving solutions — for different purposes — for the multi-antenna Gaussian channel with feedback. This insight further elucidates an interesting relationship between interactive decision theory problems and the theory of optimal transportation. We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of σ-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365―393]. A similar result is obtained also in the continuous time setting. The proofs are based on an ergodic theorem for Markov chains in random environments in a general state space. The nonlinear filter for an ergodic signal observed in white noise is said to achieve maximal accuracy if the stationary filtering error vanishes as the signal to noise ratio diverges. We give a general characterization of the maximal accuracy property in terms of various systems theoretic notions. When the signal state space is a finite set explicit necessary and sufficient conditions are obtained, while the linear Gaussian case reduces to a classic result of Kwakernaak and Sivan [IEEE Trans. Automatic Control, AC-17 (1972), pp. 79-86].