A decision maker records measurements of a finite-state Markov chain corrupted by noise. The goal is to decide when the Markov chain hits a specific target state. The decision maker can choose from a finite set of sampling intervals to pick the next time to look at the Markov chain. The aim is to optimize an objective comprising of false alarm, delay cost and cumulative measurement sampling cost. Taking more frequent measurements yields accurate estimates but incurs a higher measurement cost. Making an erroneous decision too soon incurs a false alarm penalty. Waiting too long to declare the target state incurs a delay penalty. What is the optimal sequential strategy for the decision maker? The paper shows that under reasonable conditions, the optimal strategy has the following intuitive structure: when the Bayesian estimate (posterior distribution) of the Markov chain is away from the target state, look less frequently; while if the posterior is close to the target state, look more frequently. Bounds are derived for the optimal strategy. Also the achievable optimal cost of the sequential detector as a function of transition dynamics and observation distribution is analyzed. The sensitivity of the optimal achievable cost to parameter variations is bounded in terms of the Kullback divergence. To prove the results in this paper, novel stochastic dominance results on the Bayesian filtering recursion are derived. The formulation in this paper generalizes quickest time change detection to consider optimal sampling and also yields useful results in sensor scheduling (active sensing).
We also refer to the seminal work of Moustakides (see @cite_13 and references therein) in event triggered sampling. Quickest detection has been studied widely, see @cite_21 @cite_6 and references therein. We have considered recently a POMDP approach to quickest detection with social learning @cite_19 and non-linear penalties @cite_3 and phase-distributed change times. However, in these papers, there is only one continue and one stop action. The results in the current paper are considerably more general due to the propagation of different dynamics for the multiple continue actions. A useful feature of the lattice programming approach @cite_5 @cite_9 @cite_15 used in this paper is that the results apply to general observation noise distributions (Gaussians, exponentials, discrete memoryless channels) and multiple state Markov chains. Also, the results proved here are valid for finite sample sizes and no asymptotic approximations in signal to noise ratio are used.
@cite_9 @cite_21 @cite_3 @cite_6 @cite_19 @cite_5 @cite_15 @cite_13
2102134363 2537629200 2098662489 2154153158
We show that the optimal decision policy for several types of Bayesian sequential detection problems has a threshold switching curve structure on the space of posterior distributions. This is established by using lattice programming and stochastic orders in a partially observed Markov decision process (POMDP) framework. A stochastic gradient algorithm is presented to estimate the optimal linear approximation to this threshold curve. We illustrate these results by first considering quickest time detection with phase-type distributed change time and a variance stopping penalty. Then it is proved that the threshold switching curve also arises in several other Bayesian decision problems such as quickest transient detection, exponential delay (risk-sensitive) penalties, stopping time problems in social learning, and multi-agent scheduling in a changing world. Using Blackwell dominance, it is shown that for dynamic decision making problems, the optimal decision policy is lower bounded by a myopic policy. Finally, it is shown how the achievable cost of the optimal decision policy varies with change time distribution by imposing a partial order on transition matrices. This paper studies the estimation of a high-dimensional vector signal where the observation is a known "sparse" linear transformation of the signal corrupted by additive Gaussian noise. A paradigm of such a linear system is code-division multiple access (CDMA) channel with sparse spreading matrix. Assuming a "semi-regular" ensemble of sparse matrix linear transformations, where the bi-partite graph describing the system is asymptotically cycle-free, it is shown that belief propagation (BP) achieves the minimum mean-square error (MMSE) in estimating the transformation of the input vector in the large-system limit. The result holds regardless of the the distribution and power of the input symbols. Furthermore, the mean squared error of estimating each symbol of the input vector using BP is proved to be equal to the MMSE of estimating the same symbol through a scalar Gaussian channel with some degradation in the signal-to-noise ratio (SNR). The degradation, called the efficiency, is determined from a fixed-point equation due to Guo and Verdu, which is a generalization of Tanaka's formula to arbitrary prior distributions. Recently, it was shown that it is possible to develop exact sampling schemes for a large class of parametric nonbandlimited signals, namely certain signals of finite rate of innovation. A common feature of such signals is that they have a finite number of degrees of freedom per unit of time and can be reconstructed from a finite number of uniform samples. In order to prove sampling theorems, considered the case of deterministic, noiseless signals and developed algebraic methods that lead to perfect reconstruction. However, when noise is present, many of those schemes can become ill-conditioned. In this paper, we revisit the problem of sampling and reconstruction of signals with finite rate of innovation and propose improved, more robust methods that have better numerical conditioning in the presence of noise and yield more accurate reconstruction. We analyze, in detail, a signal made up of a stream of Diracs and develop algorithmic tools that will be used as a basis in all constructions. While some of the techniques have been already encountered in the spectral estimation framework, we further explore preconditioning methods that lead to improved resolution performance in the case when the signal contains closely spaced components. For classes of periodic signals, such as piecewise polynomials and nonuniform splines, we propose novel algebraic approaches that solve the sampling problem in the Laplace domain, after appropriate windowing. Building on the results for periodic signals, we extend our analysis to finite-length signals and develop schemes based on a Gaussian kernel, which avoid the problem of ill-conditioning by proper weighting of the data matrix. Our methods use structured linear systems and robust algorithmic solutions, which we show through simulation results. This paper considers the reconstruction of structured-sparse signals from noisy linear observations. In particular, the support of the signal coefficients is parameterized by hidden binary pattern, and a structured probabilistic prior (e.g., Markov random chain field tree) is assumed on the pattern. Exact inference is discussed and an approximate inference scheme, based on loopy belief propagation (BP), is proposed. The proposed scheme iterates between exploitation of the observation-structure and exploitation of the pattern-structure, and is closely related to noncoherent turbo equalization, as used in digital communication receivers. An algorithm that exploits the observation structure is then detailed based on approximate message passing ideas. The application of EXIT charts is discussed, and empirical phase transition plots are calculated for Markov-chain structured sparsity.1