The discrete wavelet transform can be found at the heart of many image-processing algorithms. Until now, the transform on general-purpose processors (CPUs) was mostly computed using a separable lifting scheme. As the lifting scheme consists of a small number of operations, it is preferred for processing using single-core CPUs. However, considering a parallel processing using multi-core processors, this scheme is inappropriate due to a large number of steps. On such architectures, the number of steps corresponds to the number of points that represent the exchange of data. Consequently, these points often form a performance bottleneck. Our approach appropriately rearranges calculations inside the transform, and thereby reduces the number of steps. In other words, we propose a new scheme that is friendly to parallel environments. When evaluating on multi-core CPUs, we consistently overcome the original lifting scheme. The evaluation was performed on 61-core Intel Xeon Phi and 8-core Intel Xeon processors.
Usually, the 2-D transform @cite_4 is defined as the tensor product of 1-D transforms. Unlike the 1-D case, the 2-D transform splits the input signal into a quadruple of wavelet coefficients (LL, HL, LH, and HH). To describe 2-D matrices, the predict and update operators must be extended into two dimensions. Considering the separable lifting scheme, the predict and update lifting steps can be applied in both directions sequentially. It should be noted that the horizontal and vertical steps can be arbitrarily interleaved. The 2-D lifting then follow from a sequence | . | . | . | . Note the synchronization @math before the matrices. As the sequence can be hard to imagine, the individual matrices are illustrated in Figure for the CDF 5 3 wavelet @cite_0 . For multiple lifting pairs, the scheme is separately applied to each such pair. Recall that the separable lifting scheme has the smallest possible number of arithmetic operations and the highest number of steps.
@cite_0 @cite_4
2098914003 2132984323
In this paper, we introduce several new schemes for calculation of discrete wavelet transforms of images. These schemes reduce the number of steps and, as a consequence, allow to reduce the number of synchronizations on parallel architectures. As an additional useful property, the proposed schemes can reduce also the number of arithmetic operations. The schemes are primarily demonstrated on CDF 5 3 and CDF 9 7 wavelets employed in JPEG 2000 image compression standard. However, the presented method is general, and it can be applied on any wavelet transform. As a result, our scheme requires only two memory barriers for 2-D CDF 5 3 transform compared to four barriers in the original separable form or three barriers in the non-separable scheme recently published. Our reasoning is supported by exhaustive experiments on high-end graphics cards. Widespread use of wavelet transforms as in JPEG2000 demands efficient implementations on general purpose computers as well as dedicated hardware. The increasing availability of SIMD technologies is a great challenge since efficient SIMD parallelizations are not trivial. This work presents a parallelized 2D wavelet transform following a single-loop approach, i.e. a loop fusion of the lifting steps of horizontal filtering, and interleaving horizontal and vertical filtering for optimal temporal locality. In this way, each input value is read only once and each output value is written once without subsequent updates. Such an approach turns out to be a necessary basis for an efficient SIMD parallelization. Results are obtained on a general purpose processor with a 4-fold single-precision SIMD extension. Speedups of about 3.7 due to the use of SIMD, 2.55 due to the single-loop approach and up to 6 due to cache effects for pathologic data sizes are obtained, giving total speedups of up to 56. A novel approach to 2-D single-loop wavelet lifting with compatibility to JPEG 2000 is presented in this paper. A newly developed 2-D core of CDF 5 3 wavelet filter is presented that, using a new sequence of operations, simplify the design. Moreover, the proposed approach, that uses one pass for 2-D transform, directly produces final output and reduces significantly the need for storing intermediate results into memory. All the proposed structures can be efficiently pipelined in hardware. This paper describes the proposed approach, its implementation in FPGA, cost of such implementation, and brings an experimental evaluation as well as discussion of the features of the approach. This article is essentially tutorial in nature. We show how any discrete wavelet transform or two band subband filtering with finite filters can be decomposed into a finite sequence of simple filtering steps, which we call lifting steps but that are also known as ladder structures. This decomposition corresponds to a factorization of the polyphase matrix of the wavelet or subband filters into elementary matrices. That such a factorization is possible is well-known to algebraists (and expressed by the formula SL(n; R(Z, z-I)) = E(n; R(z, z-l))); it is also used in linear systems theory in the electrical engineering community. We present here a self-contained derivation, building the decomposition from basic principles such as the Euclidean algorithm, with a focus on applying it to wavelet filtering. This factorization provides an alternative for the lattice factorization, with the advantage that it can also be used in the biorthogonal, i.e., non-unitary case. Like the lattice factorization, the decomposition presented here asymptotically reduces the computational complexity of the transform by a factor two. It has other applications, such as the possibility of defining a wavelet-like transform that maps integers to integers. Abstract We present the lifting scheme, a new idea for constructing compactly supported wavelets with compactly supported duals. The lifting scheme uses a simple relationship between all multiresolution analyses with the same scaling function. It isolates the degrees of freedom remaining after fixing the biorthogonality relations. Then one has full control over these degrees of freedom to custom design the wavelet for a particular application. The lifting scheme can also speed up the fast wavelet transform. We illustrate the use of the lifting scheme in the construction of wavelets with interpolating scaling functions. We present a novel method to implement lifting based wavelet transforms on general graphs. The detail and approximation coefficients computed from this graph transform can be interpreted similarly to their counterparts in standard signal processing process. Our approach is based on partitioning all nodes in the graph into two sets, containing "even" and "odd" nodes, respectively. Then, as in standard lifting, nodes of one parity are used to predict update those of the other. We discuss the even-odd assignment problem on the graph and provide a solution that is well suited to construct the transform. As an example we discuss how our transform could be used in a denoising application. I. I NTRODUCTION An increasing number of applications require processing of signals defined on weighted graphs. While wavelets provide a flexible tool for signal processing in the classical setting of regular domains, the existing graph wavelet constructions are less flexible - they are guided solely by the structure of the underlying graph and do not take directly into consideration the particular class of signals to be processed. This paper introduces a machine learning framework for constructing graph wavelets that can sparsely represent a given class of signals. Our construction uses the lifting scheme, and is based on the observation that the recurrent nature of the lifting scheme gives rise to a structure resembling a deep auto-encoder network. Particular properties that the resulting wavelets must satisfy determine the training objective and the structure of the involved neural networks. The training is unsupervised, and is conducted similarly to the greedy pre-training of a stack of auto-encoders. After training is completed, we obtain a linear wavelet transform that can be applied to any graph signal in time and memory linear in the size of the graph. Improved sparsity of our wavelet transform for the test signals is confirmed via experiments both on synthetic and real data. With the start of the widespread use of discrete wavelet transform in image processing, the need for its efficient implementation is becoming increasingly more important. This work presents several novel SIMD-vectorized algorithms of 2-D discrete wavelet transform, using a lifting scheme. At the beginning, a stand-alone core of an already known single-loop approach is extracted. This core is further simplified by an appropriate reorganization of operations. Furthermore, the influence of the CPU cache on a 2-D processing order is examined. Finally, SIMD-vectorizations and parallelizations of the proposed approaches are evaluated. The best of the proposed algorithms scale almost linearly with the number of threads. For all of the platforms used in the tests, these algorithms are significantly faster than other known methods, as shown in the experimental sections of the paper. The 2-D discrete wavelet transform (DWT) can be found in the heart of many image-processing algorithms. Until recently, several studies have compared the performance of such transform on various shared-memory parallel architectures, especially on graphics processing units (GPUs). All these studies, however, considered only separable calculation schemes. We show that corresponding separable parts can be merged into non-separable units, which halves the number of steps. In addition, we introduce an optional optimization approach leading to a reduction in the number of arithmetic operations. The discussed schemes were adapted on the OpenCL framework and pixel shaders, and then evaluated using GPUs of two biggest vendors. We demonstrate the performance of the proposed non-separable methods by comparison with existing separable schemes. The non-separable schemes outperform their separable counterparts on numerous setups, especially considering the pixel shaders. A new transform is proposed that derives the overcomplete discrete wavelet transform (ODWT) subbands from the critically sampled DWT subbands (complete representation). This complete-to-overcomplete DWT (CODWT) has certain advantages in comparison to the conventional approach that performs the inverse DWT to reconstruct the input signal, followed by the a spl grave -trous or the lowband shift algorithm. Specifically, the computation of the input signal is not required. As a result, the minimum number of downsampling operations is performed and the use of upsampling is avoided. The proposed CODWT computes the ODWT subbands by using a set of prediction-filter matrices and filtering-and-downsampling operators applied to the DWT. This formulation demonstrates a clear separation between the single-rate and multirate components of the transform. This can be especially significant when the CODWT is used in resource-constrained environments, such as resolution-scalable image and video codecs. To illustrate the applicability of the proposed transform in these emerging applications, a new scheme for the transform-calculation is proposed, and existing coding techniques that benefit from its usage are surveyed. The analysis of the proposed CODWT in terms of arithmetic complexity and delay reveals significant gains as compared with the conventional approach. t. Below we will give an algorithm which computes the coefficients of the product of two square matrices A and B of order n from the coefficients of A and B with tess than 4 . 7 n l°g7 arithmetical operations (all logarithms in this paper are for base 2, thus tog 7 2.8; the usual method requires approximately 2n 3 arithmetical operations). The algorithm induces algorithms for invert ing a matr ix of order n, solving a system of n linear equations in n unknowns, comput ing a determinant of order n etc. all requiring less than const n l°g 7 arithmetical operations. This fact should be compared with the result of KLYUYEV and KOKOVKINSHCHERBAK [1 ] tha t Gaussian elimination for solving a system of l inearequations is optimal if one restricts oneself to operations upon rows and columns as a whole. We also note tha t WlNOGRAD [21 modifies the usual algorithms for matr ix multiplication and inversion and for solving systems of linear equations, trading roughly half of the multiplications for additions and subtractions. I t is a pleasure to thank D. BRILLINGER for inspiring discussions about the present subject and ST. COOK and B. PARLETT for encouraging me to write this paper. 2. We define algorithms e , which mult iply matrices of order m2 , by induction on k: , 0 is the usual algorithm, for matr ix multiplication (requiring m a multiplications and m 2 ( m t) additions), e ,k already being known, define , +t as follows: If A, B are matrices of order m 2 k to be multiplied, write 1 Model Problem and Its Discretization.- 1.1 Conservation Laws.- 1.2 Finite Volume Methods.- 2 Multiscale Setting.- 2.1 Hierarchy of Meshes.- 2.2 Motivation.- 2.3 Box Wavelet.- 2.3.1 Box Wavelet on a Cartesian Grid Hierarchy.- 2.3.2 Box Wavelet on an Arbitrary Nested Grid Hierarchy.- 2.4 Change of Stable Completion.- 2.5 Box Wavelet with Higher Vanishing Moments.- 2.5.1 Definition and Construction.- 2.5.2 A Univariate Example.- 2.5.3 A Remark on Compression Rates.- 2.6 Multiscale Transformation.- 3 Locally Refined Spaces.- 3.1 Adaptive Grid and Significant Details.- 3.2 Grading.- 3.3 Local Multiscale Transformation.- 3.4 Grading Parameter.- 3.5 Locally Uniform Grids.- 3.6 Algorithms: Encoding, Thresholding, Grading, Decoding.- 3.7 Conservation Property.- 3.8 Application to Curvilinear Grids.- 4 Adaptive Finite Volume Scheme.- 4.1 Construction.- 4.1.1 Strategies for Local Flux Evaluation.- 4.1.2 Strategies for Prediction of Details.- 4.2 A gorithms: Initial data, Prediction, Fluxes and Evolution.- 5 Error Analysis.- 5.1 Perturbation Error.- 5.2 Stability of Approximation.- 5.3 Reliability of Prediction.- 6 Data Structures and Memory Management.- 6.1 Algorithmic Requirements and Design Criteria.- 6.2 Hashing.- 6.3 Data Structures.- 7 Numerical Experiments.- 7.1 Parameter Studies.- 7.1.1 Test Configurations.- 7.1.2 Discretization.- 7.1.3 Computational Complexity and Stability.- 7.1.4 Hash Parameters.- 7.2 Real World Application.- 7.2.1 Configurations.- 7.2.2 Discretization.- 7.2.3 Discussion of Results.- A Plots of Numerical Experiments.- B The Context of Biorthogonal Wavelets.- B.1 General Setting.- B.1.1 Multiscale Basis.- B.1.2 Stable Completion.- B.1.3 Multiscale Transformation.- B.2 Biorthogonal Wavelets of the Box Function.- B.2.1 Haar Wavelets.- B.2.2 Biorthogonal Wavelets on the Real Line.- References.- List of Figures.- List of Tables.- Notation. Lifted message passing algorithms exploit repeated structure within a given graphical model to answer queries efficiently. Given evidence, they construct a lifted network of supernodes and superpotentials corresponding to sets of nodes and potentials that are indistinguishable given the evidence. Recently, efficient algorithms were presented for updating the structure of an existing lifted network with incremental changes to the evidence. In the inference stage, however, current algorithms need to construct a separate lifted network for each evidence case and run a modified message passing algorithm on each lifted network separately. Consequently, symmetries across the inference tasks are not exploited. In this paper, we present a novel lifted message passing technique that exploits symmetries across multiple evidence cases. The benefits of this multi-evidence lifted inference are shown for several important AI tasks such as computing personalized PageRanks and Kalman filters via multievidence lifted Gaussian belief propagation. Let G be a graph on n vertices. A 2-lift of G is a graph H on 2n vertices, with a covering map � : H → G. It is not hard to see that all eigenvalues of G are also eigenvalues of H. In addition, H has n “new” eigenvalues. We conjecture that every d-regular graph has a 2lift such that all new eigenvalues are in the range [−2 √ d − 1,2 √ d − 1] (If true, this is tight , e.g. by the Alon-Boppana bound). Here we show that every graph of maximal degree d has a 2-lift such that all Abstract Invertible wavelet transforms that map integers to integers have important applications in lossless coding. In this paper we present two approaches to build integer to integer wavelet transforms. The first approach is to adapt the precoder of Laroiaet al,which is used in information transmission; we combine it with expansion factors for the high and low pass band in subband filtering. The second approach builds upon the idea of factoring wavelet transforms into so-called lifting steps. This allows the construction of an integer version of every wavelet transform. Finally, we use these approaches in a lossless image coder and compare the results to those given in the literature. One. Introduction: On the Absence of Geometry from Morphometrics.- First Part. The Measurement of Biological Shape.- Two. Shapes and Measures of Shape.- A. Properties of the Euclidean Plane and Euclidean Space.- B. Outlines and Homologous Landmarks.- C. Definitions of Shape, Shape Change, Shape Measurement.- D. Shapes as Data.- Three. Critique of an Applied Field: Conventional Cephalometrics.- A. Landmarks, Curvature, and Growth.- B. Registration.- Four. New Statistical Methods for Shape.- A. Analysis of the Tangent Angle Function.- 1. History.- 2. Sampling from the tangent angle function.- 3. Conic replacement curves and their estimation Fit of a conic to a point - Geometric interpretation - Estimator for a circle - Estimation for the general conic - Computation of the extremum -Linear constraints.- 4. Conic splining Joint conic fitting under constraint -An example - Application - Analysis of parameters.- B. Extension to Three Dimensions: A Sketch.- C. Skeletons Definition - A multivariate statistical method - Bibliographic note.- Second Part. The Measurement of Shape Change Using Biorthogonal Grids.- Five. The Study of Shape Transformation after D'Arcy Thompson.- A. The Original Method.- 1. Thompson's own work.- 2. Later examples.- 3. Difficulties.- B. Analysis of Growth Gradients.- C. Simulations.- D. Other Morphometric Schemes Vector displacements - Multivariate morphometrics.- Six. The Method of Biorthogonal Grids.- A. Representation of Affine Transformations.- B. General Lines of Growth and Biorthogonal Grids.- C. Summarizing the Grids.- Technical Note 1. Existence and Form of Biorthogonal Grids.- Technical Note 2. Interpolation from Landmark Locations and Arcs The measure of roughness - The vector space and its associated functions - Interpolation from boundary values - Interpolation from boundary values and interior points - Note on computation.- Technical Note 3. Construction of Integral Curves.- Technical Note 4. On Homologous Points.- Seven. Examples of Biorthogonal Analysis.- A. Comparison of Square and Biorthogonal Grids: Thompson's Diodon Figure.- B. Phylogeny and Ontogeny of Primate Crania.- 1. Functional craniology and craniometrics.- 2. Data for this exercise.- 3. Two types of transformations.- 4. Comparative ontogeny of the apes and man.- Eight. Future Directions for Transformation Analysis.- A. Statistical Methods The symmetric tensor field - Concordance -Linear methods.- B. Computation Other kinds of information about homology -Three dimensions.- C. Likely Applications Computed tomography - Orthodontics - Developmental biology.- Nine. Envoi.- Literature Cited. Demodulation in a random multiple access channel is considered where the signals are chosen uniformly randomly with unit energy. It is shown that by lifting (replicating) the graph of this system and randomizing the graph connections, a simple iterative cancellation demodulator achieves the same performance as an optimal symbol-by-symbol detector of the original system. The iterative detector has a complexity that is linear in the number of users, while the direct optimal approach is known to be NP-hard. However, the maximal system load of this lifted graph is limited to @math $ can go to infinity as the SNR goes to infinity. Our results apply to several well-documented system proposals, such as interleave-division multiple access, partitioned spreading, and certain forms of multiple-input multiple-output communications. The adaptive wavelet packet transform is applied to sparsify the moment matrices for the fast solution of electromagnetic integral equations. In the algorithm, a cost function is employed to adaptively select the optimal wavelet packet expansion testing functions to achieve the maximum sparsity possible in the resulting transformed system. The search for the best wavelet packet basis and the moment matrix transformation are implemented by repeated two-channel filtering of the original moment matrix with a pair of quadrature filters. It is found that the sparsified matrix has above-threshold elements that grow only as O(N sup 1.4 ) for typical scatterers. Consequently the operations to solve the transformed moment equation using the conjugate gradient method scales as O(N sup 1.4 ). The additional computational cost for carrying out the adaptive wavelet packet transform is evaluated and discussed. In this paper, a new algorithm to efficiently implement the two-dimensional lifting scheme is presented. The 1D lifting-scheme performs in-place processing of the input samples, and hence it provides reduction in memory requirements. However, for image processing (2D), in-place computation is not enough, resulting in a memory-intensive algorithm, since it has to keep the whole image in memory. We propose the use of line-by-line processing algorithm for the lifting scheme, and we address some issues on how to perform synchronization among different buffer levels, so that an implementation can be easily written. Experimental results show that, for a 5-megapixel image, our algorithm requires 200 times less memory and it is more than 3 times faster than the usual one. 1 Preliminaries.- 1.1 Introduction.- 1.2 Measures and Functions.- 1.3 Weak Topologies.- 1.4 Convergence of Measures.- 1.5 Complements.- 1.6 Notes.- I Markov Chains and Ergodicity.- 2 Markov Chains and Ergodic Theorems.- 2.1 Introduction.- 2.2 Basic Notation and Definitions.- 2.3 Ergodic Theorems.- 2.4 The Ergodicity Property.- 2.5 Pathwise Results.- 2.6 Notes.- 3 Countable Markov Chains.- 3.1 Introduction.- 3.2 Classification of States and Class Properties.- 3.3 Limit Theorems.- 3.4 Notes.- 4 Harris Markov Chains.- 4.1 Introduction.- 4.2 Basic Definitions and Properties.- 4.3 Characterization of Harris recurrence.- 4.4 Sufficient Conditions for P.H.R.- 4.5 Harris and Doeblin Decompositions.- 4.6 Notes.- 5 Markov Chains in Metric Spaces.- 5.1 Introduction.- 5.2 The Limit in Ergodic Theorems.- 5.3 Yosida's Ergodic Decomposition.- 5.4 Pathwise Results.- 5.5 Proofs.- 5.6 Notes.- 6 Classification of Markov Chains via Occupation Measures.- 6.1 Introduction.- 6.2 A Classification.- 6.3 On the Birkhoff Individual Ergodic Theorem.- 6.4 Notes.- II Further Ergodicity Properties.- 7 Feller Markov Chains.- 7.1 Introduction.- 7.2 Weak-and Strong-Feller Markov Chains.- 7.3 Quasi Feller Chains.- 7.4 Notes.- 8 The Poisson Equation.- 8.1 Introduction.- 8.2 The Poisson Equation.- 8.3 Canonical Pairs.- 8.4 The Cesaro-Averages Approach.- 8.5 The Abelian Approach.- 8.6 Notes.- 9 Strong and Uniform Ergodicity.- 9.1 Introduction.- 9.2 Strong and Uniform Ergodicity.- 9.3 Weak and Weak Uniform Ergodicity.- 9.4 Notes.- III Existence and Approximation of Invariant Probability Measures.- 10 Existence of Invariant Probability Measures.- 10.1 Introduction and Statement of the Problems.- 10.2 Notation and Definitions.- 10.3 Existence Results.- 10.4 Markov Chains in Locally Compact Separable Metric Spaces.- 10.5 Other Existence Results in Locally Compact Separable Metric Spaces.- 10.6 Technical Preliminaries.- 10.7 Proofs.- 10.8 Notes.- 11 Existence and Uniqueness of Fixed Points for Markov Operators.- 11.1 Introduction and Statement of the Problems.- 11.2 Notation and Definitions.- 11.3 Existence Results.- 11.4 Proofs.- 11.5 Notes.- 12 Approximation Procedures for Invariant Probability Measures.- 12.1 Introduction.- 12.2 Statement of the Problem and Preliminaries.- 12.3 An Approximation Scheme.- 12.4 A Moment Approach for a Special Class of Markov Chains.- 12.5 Notes.