The theory of sparse stochastic processes offers a broad class of statistical models to study signals. In this framework, signals are represented as realizations of random processes that are solution of linear stochastic differential equations driven by white L 'evy noises. Among these processes, generalized Poisson processes based on compound-Poisson noises admit an interpretation as random L-splines with random knots and weights. We demonstrate that every generalized L 'evy process-from Gaussian to sparse-can be understood as the limit in law of a sequence of generalized Poisson processes. This enables a new conceptual understanding of sparse processes and suggests simple algorithms for the numerical generation of such objects.
 In a companion paper (see Self-Similarity: Part I-Splines and Operators), we characterized the class of scale-invariant convolution operators: the generalized fractional derivatives of order gamma. We used these operators to specify regularization functionals for a series of Tikhonov-like least-squares data fitting problems and proved that the general solution is a fractional spline of twice the order. We investigated the deterministic properties of these smoothing splines and proposed a fast Fourier transform (FFT)-based implementation. Here, we present an alternative stochastic formulation to further justify these fractional spline estimators. As suggested by the title, the relevant processes are those that are statistically self-similar; that is, fractional Brownian motion (fBm) and its higher order extensions. To overcome the technical difficulties due to the nonstationary character of fBm, we adopt a distributional formulation due to Gel'fand. This allows us to rigorously specify an innovation model for these fractal processes, which rests on the property that they can be whitened by suitable fractional differentiation. Using the characteristic form of the fBm, we then derive the conditional probability density function (PDF) p(BH(t)|Y), where Y= BH(k)+n[k] kisinZ are the noisy samples of the fBm BH(t) with Hurst exponent H. We find that the conditional mean is a fractional spline of degree 2H, which proves that this class of functions is indeed optimal for the estimation of fractal-like processes. The result also yields the optimal [minimum mean-square error (MMSE)] parameters for the smoothing spline estimator, as well as the connection with kriging and Wiener filtering We introduce an extended family of continuous-domain stochastic models for sparse, piecewise-smooth signals. These are specified as solutions of stochastic differential equations, or, equivalently, in terms of a suitable innovation model; the latter is analogous conceptually to the classical interpretation of a Gaussian stationary process as filtered white noise. The two specific features of our approach are 1) signal generation is driven by a random stream of Dirac impulses (Poisson noise) instead of Gaussian white noise, and 2) the class of admissible whitening operators is considerably larger than what is allowed in the conventional theory of stationary processes. We provide a complete characterization of these finite-rate-of-innovation signals within Gelfand's framework of generalized stochastic processes. We then focus on the class of scale-invariant whitening operators which correspond to unstable systems. We show that these can be solved by introducing proper boundary conditions, which leads to the specification of random, spline-type signals that are piecewise-smooth. These processes are the Poisson counterpart of fractional Brownian motion; they are nonstationary and have the same 1 ω-type spectral signature. We prove that the generalized Poisson processes have a sparse representation in a wavelet-like basis subject to some mild matching condition. We also present a limit example of sparse process that yields a MAP signal estimator that is equivalent to the popular TV-denoising algorithm. "...a blend of erudition (fascinating and sometimes obscure historical minutiae abound), popularization (mathematical rigor is relegated to appendices) and exposition (the reader need have little knowledge of the fields involved) ...and the illustrations include many superb examples of computer graphics that are works of art in their own right." Nature   Statistical analysis of images reveals two interesting properties: (i) invariance of image statistics to scaling of images, and (ii) non-Gaussian behavior of image statistics, i.e. high kurtosis, heavy tails, and sharp central cusps. In this paper we review some recent results in statistical modeling of natural images that attempt to explain these patterns. Two categories of results are considered: (i) studies of probability models of images or image decompositions (such as Fourier or wavelet decompositions), and (ii) discoveries of underlying image manifolds while restricting to natural images. Applications of these models in areas such as texture analysis, image classification, compression, and denoising are also considered. Preface Notation What Is Pattern Theory? The Manifesto of Pattern Theory The Basic Types of Patterns Bayesian Probability Theory: Pattern Analysis and Pattern Synthesis English Text and Markov Chains Basics I: Entropy and Information Measuring the n-gram Approximation with Entropy Markov Chains and the n-gram Models Words Word Boundaries via Dynamic Programming and Maximum Likelihood Machine Translation via Bayes' Theorem Exercises Music and Piece wise Gaussian Models Basics III: Gaussian Distributions Basics IV: Fourier Analysis Gaussian Models for Single Musical Notes Discontinuities in One-Dimensional Signals The Geometric Model for Notes via Poisson Processes Related Models Exercises Character Recognition and Syntactic Grouping Finding Salient Contours in Images Stochastic Models of Contours The Medial Axis for Planar Shapes Gestalt Laws and Grouping Principles Grammatical Formalisms Exercises Contents Image Texture, Segmentation and Gibbs Models Basics IX: Gibbs Fields (u + v)-Models for Image Segmentation Sampling Gibbs Fields Deterministic Algorithms to Approximate the Mode of a Gibbs Field Texture Models Synthesizing Texture via Exponential Models Texture Segmentation Exercises Faces and Flexible Templates Modeling Lighting Variations Modeling Geometric Variations by Elasticity Basics XI: Manifolds, Lie Groups, and Lie Algebras Modeling Geometric Variations by Metrics on Diff Comparing Elastic and Riemannian Energies Empirical Data on Deformations of Faces The Full Face Model Appendix: Geodesics in Diff and Landmark Space Exercises Natural Scenes and their Multiscale Analysis High Kurtosis in the Image Domain Scale Invariance in the Discrete and Continuous Setting The Continuous and Discrete Gaussian Pyramids Wavelets and the "Local" Structure of Images Distributions Are Needed Basics XIII: Gaussian Measures on Function Spaces The Scale -Rotation- and Translation-Invariant Gaussian Distribution Mode lII: Images Made Up of Independent Objects Further Models Appendix: A Stability Property of the Discrete Gaussian Pyramid Exercises Bibliography Index 1. Introduction 2. Roadmap to the book 3. Mathematical context and background 4. Continuous-domain innovation models 5. Operators and their inverses 6. Splines and wavelets 7. Sparse stochastic processes 8. Sparse representations 9. Infinite divisibility and transform-domain statistics 10. Recovery of sparse signals 11. Wavelet-domain methods 12. Conclusion Appendix A. Singular integrals Appendix B. Positive definiteness Appendix C. Special functions. Our study of fractal landscapes departs from the simplest but yet effective model of fractional Brownian motion and explores its two-dimensional (2-D) extensions. We focus on the ability to introduce anisotropy in this model, and we are also interested in considering its discrete-space counterparts. We then move towards other multifractional and multifractal models providing more degrees of freedom for fitting complex 2-D fields. We note that many of the models and processing are implemented in FracLab, a software MATLAB Scilab toolbox for fractal processing of signals and images. The idea of using statistical inference for analyzing and understanding images has been used for at least 20 years, going back, for instance, to the work of Grenander [Gr] and Cooper [Co]. To apply these techniques, one needs, of course, a probabilistic model for some class of images or some class of structures present in images. Many models of this type have been introduced. There are stochastic models for image textures [GGGD], [ZMW], for contours in images [Mu], [GCK], for the decomposition of an image into regions [G-G], [M-S], for disparity maps, for grammatical parsing of shapes [Fu], for template matching, for speci c tasks such as face recognition [HGYGM]. The common framework for all these studies is to describe some class of images I(x; y) by means of a set of auxiliary variables fx g representing the salient structures in the images, e.g. edges, texture statistics, inferred depth values or relations, illumination features, medial axes or shape features, locations of key points such as eyes in a face, labels (as in character recognition), etc. Then i) a prior probability model for the hidden' variables p(fx g) and ii) an imagingmodel p(Ijfx g) for I, given the hidden variables, are de ned. Finally, an image is analyzed using Bayes's rule p(fx gjI) p(Ijfx g)p(fx g)