Motion estimation is an important component of video codecs and various applications in computer vision. Especially in video compression the compact representation of motion fields is crucial, as modern video codecs use them for inter frame prediction. In recent years compression methods relying on diffusion-based inpainting have been becoming an increasingly competitive alternative to classical transform-based codecs. They perform particularly well on piecewise smooth data, suggesting that motion fields can be efficiently represented by such approaches. However, they have so far not been used for the compression of motion data. Therefore, we assess the potential of flow field compression based on homogeneous diffusion with a specifically designed new framework: Our codec stores only a few representative flow vectors and reconstructs the flow field with edge-aware homogeneous diffusion inpainting. Additionally stored edge data thereby ensure the accurate representation of discontinuities in the flow field. Our experiments show that this approach can outperform state-of-the-art codecs such as JPEG2000 and BPG HEVC intra.
Compression methods with diffusion-based inpainting were introduced by Gali ' c et al @cite_3 in 2005. Their method stores only a few selected pixels of an image and reconstructs missing data with edge-enhancing anisotropic diffusion @cite_10 . The R-EED algorithm of @cite_22 improves this idea with an efficient tree structure to adaptively encode mask pixels and can beat the quality of JPEG2000.
@cite_10 @cite_22 @cite_3
2157028296 2952830092 1586517188 2793519528
While methods based on partial differential equations (PDEs) and variational techniques are powerful tools for denoising and inpainting digital images, their use for image compression was mainly focussing on pre- or postprocessing so far. In our paper we investigate their potential within the decoding step. We start with the observation that edge-enhancing diffusion (EED), an anisotropic nonlinear diffusion filter with a diffusion tensor, is well-suited for scattered data interpolation: Even when the interpolation data are very sparse, good results are obtained that respect discontinuities and satisfy a maximum–minimum principle. This property is exploited in our studies on PDE-based image compression. We use an adaptive triangulation method based on B-tree coding for removing less significant pixels from the image. The remaining points serve as scattered interpolation data for the EED process. They can be coded in a compact and elegant way that reflects the B-tree structure. Our experiments illustrate that for high compression rates and non-textured images, this PDE-based approach gives visually better results than the widely-used JPEG coding. We propose a method for lossy image compression based on recurrent, convolutional neural networks that outperforms BPG (4:2:0 ), WebP, JPEG2000, and JPEG as measured by MS-SSIM. We introduce three improvements over previous research that lead to this state-of-the-art result. First, we show that training with a pixel-wise loss weighted by SSIM increases reconstruction quality according to several metrics. Second, we modify the recurrent architecture to improve spatial diffusion, which allows the network to more effectively capture and propagate image information through the network's hidden state. Finally, in addition to lossless entropy coding, we use a spatially adaptive bit allocation algorithm to more efficiently use the limited number of bits to encode visually complex image regions. We evaluate our method on the Kodak and Tecnick image sets and compare against standard codecs as well recently published methods based on deep neural networks. Although widely used standards such as JPEG and JPEG 2000 exist in the literature, lossy image compression is still a subject of ongoing research. (2008) have shown that compression based on edge-enhancing anisotropic diffusion can outperform JPEG for medium to high compression ratios when the interpolation points are chosen as vertices of an adaptive triangulation. In this paper we demonstrate that it is even possible to beat the quality of the much more advanced JPEG 2000 standard when one uses subdivisions on rectangles and a number of additional optimisations. They include improved entropy coding, brightness rescaling, diffusivity optimisation, and interpolation swapping. Experiments on classical test images are presented that illustrate the potential of our approach. This paper proposes a lenslet image compression method scalable from low bitrates to fully lossless. The subaperture images are split into two sets: a set of reference views, encoded by a standard lossy or lossless compressor, and the set of dependent views, which are reconstructed by sparse prediction from the reference set using the geometrical information from the depth map. The set of reference views may contain all views and all views may also be dependent views, in which case the sparse predictive stage does not reconstruct from scratch the views, but it refines in a sequential order all views by combining in an optimal way the information about the same region existing in neighbor views. The encoder transmits to the decoder a segmented version of the scene depthmap, the encoded versions of the reference views, displacements for each region from the central view to each of the dependent views, and finally the sparse predictors for each region and each dependent view. The scheme can be configured to ensure random access to the dependent views, while the reference views are compressed in a backward compatible way, e.g., using JPEG 2000. The experimental results show performance better than that of the baseline standard compressor used, JPEG 2000.