A dynamical model consists of a continuous self-map @math of a compact state space @math and a continuous observation function @math . This paper considers the fitting of a parametrized family of dynamical models to an observed real-valued stochastic process using empirical risk minimization. The limiting behavior of the minimum risk parameters is studied in a general setting. We establish a general convergence theorem for minimum risk estimators and ergodic observations. We then study conditions under which empirical risk minimization can effectively separate the signal from the noise in an additive observational noise model. The key, necessary condition in the latter results is that the family of dynamical models has limited complexity, which is quantified through a notion of entropy for families of infinite sequences. Close connections between entropy and limiting average mean widths for stationary processes are established.
Some of Furstenberg's original results are extended in recent work of Lev, Peled, and Peres @cite_10 . Given an infinite sequence equal to a target signal plus noise, they consider the problem of detecting whether the signal is non-zero, and the problem of recovering the signal from the given sequence. Target sequences are assumed to belong to a known family (as in @cite_53 ), and their analysis places no restrictions (beyond measurability) on the detection and filtering procedures, which can be functions of the entire sequence of observations. Finally, we mention that statistical inference in the context of dynamical systems has been considered in a variety of subject areas; see the survey @cite_49 for a broad overview and references. Dynamical systems in the observational noise setting have been studied in @cite_24 @cite_22 @cite_31 , and statistical prediction in the context of dynamical systems has been considered in @cite_20 @cite_14 @cite_12 @cite_16 .
@cite_14 @cite_22 @cite_53 @cite_24 @cite_49 @cite_31 @cite_16 @cite_10 @cite_12 @cite_20
2071091379 2128007683 2571527823 2953338282
The problem of extracting a signal x n from a noise-corrupted time series y n = x n + e n is considered. The signal x n is assumed to be generated by a discrete-time, deterministic, chaotic dynamical system F, in particular, x n = F n (x 0 ), where the initial point x 0 is assumed to lie in a compact hyperbolic F-invariant set. It is shown that (1) if the noise sequence e n is Gaussian then it is impossible to consistently recover the signal x n , but (2) if the noise sequence consists of i.i.d. random vectors uniformly bounded by a constant δ > 0, then it is possible to recover the signal x n provided δ < 5Δ, where Δ is a separation threshold for F. A filtering algorithm for the latter situation is presented. It has been known for a while that l1-norm relaxation can in certain cases solve an under-determined system of linear equations. Recently, E. Candes ("Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Information Theory, vol. 52, no. 12, pp. 489-509, Dec. 2006) and D. Donoho ("High-dimensional centrally symmetric polytopes with neighborlines proportional to dimension," Disc. Comput. Geometry, vol. 35, no. 4, pp. 617-652, 2006) proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of nonzero elements of the unknown vector) also proportional to the length of the unknown vector such that l1-norm relaxation succeeds in solving the system. In this paper, in a large dimensional and statistical context, we determine sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations for the case of the so-called block-sparse unknown vectors considered in "On the reconstruction of block-sparse signals with an optimal number of measurements," (M. , IEEE Trans, Signal Processing, submitted for publication. We study the compressed sensing reconstruction problem for a broad class of random, band-diagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by Krzakala [30], message passing algorithms can effectively solve the reconstruction problem for spatially coupled measurements with undersampling rates close to the fraction of nonzero coordinates. We use an approximate message passing (AMP) algorithm and analyze it through the state evolution method. We give a rigorous proof that this approach is successful as soon as the undersampling rate δ exceeds the (upper) Renyi information dimension of the signal, d(pX). More precisely, for a sequence of signals of diverging dimension n whose empirical distribution converges to pX, reconstruction is with high probability successful from d(pX) n+o(n) measurements taken according to a band diagonal matrix. For sparse signals, i.e., sequences of dimension n and k(n) nonzero entries, this implies reconstruction from k(n)+o(n) measurements. For “discrete” signals, i.e., signals whose coordinates take a fixed finite set of values, this implies reconstruction from o(n) measurements. The result is robust with respect to noise, does not apply uniquely to random signals, but requires the knowledge of the empirical distribution of the signal pX. We study the compressed sensing reconstruction problem for a broad class of random, band-diagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by KrzakalaEtAl , message passing algorithms can effectively solve the reconstruction problem for spatially coupled measurements with undersampling rates close to the fraction of non-zero coordinates. We use an approximate message passing (AMP) algorithm and analyze it through the state evolution method. We give a rigorous proof that this approach is successful as soon as the undersampling rate @math exceeds the (upper) R 'enyi information dimension of the signal, @math . More precisely, for a sequence of signals of diverging dimension @math whose empirical distribution converges to @math , reconstruction is with high probability successful from @math measurements taken according to a band diagonal matrix. For sparse signals, i.e., sequences of dimension @math and @math non-zero entries, this implies reconstruction from @math measurements. For discrete' signals, i.e., signals whose coordinates take a fixed finite set of values, this implies reconstruction from @math measurements. The result is robust with respect to noise, does not apply uniquely to random signals, but requires the knowledge of the empirical distribution of the signal @math .