A dynamical model consists of a continuous self-map @math of a compact state space @math and a continuous observation function @math . This paper considers the fitting of a parametrized family of dynamical models to an observed real-valued stochastic process using empirical risk minimization. The limiting behavior of the minimum risk parameters is studied in a general setting. We establish a general convergence theorem for minimum risk estimators and ergodic observations. We then study conditions under which empirical risk minimization can effectively separate the signal from the noise in an additive observational noise model. The key, necessary condition in the latter results is that the family of dynamical models has limited complexity, which is quantified through a notion of entropy for families of infinite sequences. Close connections between entropy and limiting average mean widths for stationary processes are established.
Furstenberg's original work on joinings @cite_52 includes an application of joinings to a nonlinear filtering problem. Beyond this application, we are not aware of other uses of joinings in the literature on statistical inference. Ornstein and Weiss @cite_57 studied the estimation of a stochastic process from its samples. They proposed an inference procedure, based on matching @math -block frequencies, and characterize when it produces consistent estimates of the observed stochastic process in the @math -bar metric.
@cite_57 @cite_52
2069641973 2059735611
A series of observations ξ 1 , ξ 2 , ξ 3 ,... is presented to us and at each time n, when we have observed the first n of them, we are called upon to give our guess for what stochastic process produced the data. A universal scheme is given which, for any Bernoulli process (not necessarily independent), gives a sequence of processes that converges in a strong sense (the d 36-metric) to the real process. It is shown that in a certain sense the class of Bernoulli processes is the largest one for which such a universal scheme is possible 0. Summary. The objects of ergodic theory -measure spaces with measure-preserving transformation groups-wil l be called processes, those of topological dynamics-compact metric spaces with groups of homeomorphisms-will be called flows. We shall be concerned with what may be termed the "arithmetic" of these classes of objects. One may form products of processes and of flows, and one may also speak of factor processes and factor flows. By analogy with the integers, we may say that two processes are relatively prime if they have no non-trivial factors in common. An alternative condition is that whenever the two processes appear as factors of a third process, then their product too appears as a factor. In our theories it is unknown whether these two conditions are equivalent. We choose the second of these conditions as the more useful and refer to it as disjointness. Our first applications of the concept of disjointness are to the classification of processes and flows. It will appear that certain classes of processes (flows) may be characterized by the property of being disjoint from the members of other classes of processes (flows). For example the processes with entropy 0 are just those which are disjoint from all Bernoulli flows. Another application of disjointness of processes is to the following filtering problem. If xn and Yn represent two stationary stochastic processes, when can xn be filtered perfectly from Xn + Yn ? We will find (Part I, §9) that a sufficient condition is the disjointness of the processes in question. For flows the principal application of disjointness is to the tudy of properties of minimal sets (Part III). Consider the flow on the unit circle K = z: [zl = 1 that arises from the transformation z -- z 2. What can be said about the "size" of the minimal sets for this flow, that is, closed subsets of K invariant under z z , but not containing proper subsets with these properties. Uncountably many such minimal sets exist in K. Writing z = exp (2 ri Ean 2n), an = 0, 1, we see that this amounts to studying the mini-