The availability of affordable 3D full body reconstruction systems has given rise to free-viewpoint video (FVV) of human shapes. Most existing solutions produce temporally uncorrelated point clouds or meshes with unknown point vertex correspondences. Individually compressing each frame is ineffective and still yields to ultra-large data sizes. We present an end-to-end deep learning scheme to establish dense shape correspondences and subsequently compress the data. Our approach uses sparse set of "panoramic" depth maps or PDMs, each emulating an inward-viewing concentric mosaics. We then develop a learning-based technique to learn pixel-wise feature descriptors on PDMs. The results are fed into an autoencoder-based network for compression. Comprehensive experiments demonstrate our solution is robust and effective on both public and our newly captured datasets.
Different from mesh correspondence matching, animated mesh compression is a well-studied problem in computer graphics. State-of-the-art solutions however assume consecutive meshes have exact connectivity @cite_27 . @cite_8 @cite_26 conduct pre-segmentation on the mesh to ensure connectivity consistencies. PCA-based methods @cite_19 @cite_28 @cite_33 @cite_34 aim to identify different geometric clusters of human body (arms, hands, legs, torso, head, etc). Spatio-temporal analysis @cite_16 @cite_22 @cite_34 predict vertex trajectories for forming vertex groups. There is by far only a handful of works @cite_0 @cite_37 focusing directly on compressing uncorrelated mesh sequences, i.e., meshes without correspondences. The quality of these approach fall short compared with the ones with correspondences. Our learning based approach in contrast can robustly and accurately process uncorrelated mesh sequences and is able to achieve very high compression rate. Further, we employ the smoothness of the correspondence trajectory as well as geodesic consistencies for fine tuning our solution.
@cite_37 @cite_26 @cite_33 @cite_22 @cite_8 @cite_28 @cite_16 @cite_0 @cite_19 @cite_27 @cite_34
2017910919 2166329289 2108891011 2162839010 2066472867 2139989075 2155641907 2160793724 2091797932 2118084436
This paper proposes intra-frame and inter-frame coding algorithms for 3D mesh sequences generated by multiple cameras, which we call Time-Varying Meshes (TVMs). For this purpose, mesh segmentation into patches with patch alignment using principal component analysis (PCA) is proposed. The patches are used as minimum units to eliminate spatial and temporal correlation of the TVMs. For intraframe coding of geometry data, spectral compression using the Kirchhoff matrix was employed and that of color texture was conducted using vector quantization (VQ). The inter-frame coding of geometry data and color texture was achieved by the combination of patch-based matching and simple scalar quantization. The intra- and inter-frame coding are compared to the previous works and demonstrated promising results. This paper proposes a novel approach for mono-resolution 3D mesh compression, called TFAN (Triangle Fan-based compression). TFAN treats in a unified manner meshes of arbitrary topologies, i.e., manifold or not, oriented or not, while offering a linear computational complexity (with respect to the number of mesh vertices) for both encoding and decoding algorithms. In addition, the TFAN compressed representation is optimized for real-time decoding applications. In order to validate the proposed approach, two databases have been considered for experimentations. The first is the MPEG-4 test set, which includes over 3500 general purpose manifold meshes. The second, related to the French national project SEMANTIC-3D, includes over 4000 computer assisted design (CAD) meshes of highly irregular, non-manifold topologies. In both cases, the TFAN approach outperforms existing techniques such as MPEG-4-3DMC (3D Mesh Coding) or Touma and Gotsman, with decoding times lower by an order of magnitude at equivalent or even better levels of compression efficiency (±10p in bitrate). In addition, when applied to non-manifold 3D data, the compression performances are significantly enhanced (6–30p gain in bitrate). Due to its high compression performances the TFAN approach has been recently retained for ISO standardization, within the framework of the MPEG-4-AFX standard. Copyright © 2009 John Wiley & Sons, Ltd. The proposed TFAN (Triangle Fan-based) codec supports real-time encoding-decoding of 3D meshes with arbitrary topologies (i.e. manifold or not, oriented or not), by specifying ten topological configurations (cf. Figure 1). TFAN outperforms existing techniques such as MPEG-4-3DMC or Touma & Gotsman, with decoding times lower by an order of magnitude at equivalent or even better levels of compression efficiency. Due to its high performances, the TFAN approach has been recently retained for integration within the MPEG-4-AFX standard. We present a new geometry compression method for animations, which is based on the clustered principal component analysis (CPCA). Instead of analyzing the set of vertices for each frame, our method analyzes the set of paths for all vertices for a certain animation length. Thus, using a data-driven approach, it can identify mesh parts, that are "coherent" over time. This usually leads to a very efficient and robust segmentation of the mesh into meaningful clusters, e.g. the wings of a chicken. These parts are then compressed separately using standard principal component analysis (PCA). Each of this clusters can be compressed more efficiently with lesser PCA components compared to previous approaches. Results show, that the new method outperforms other compression schemes like pure PCA based compression or combinations with linear prediction coding, while maintaining a better reconstruction error. This is true, even if the components and weights are quantized before transmission. The reconstruction process is very simple and can be performed directly on the GPU. Dynapack exploits space-time coherence to compress the consecutive frames of the 3D animations of triangle meshes of constant connectivity. Instead of compressing each frame independently (space-only compression) or compressing the trajectory of each vertex independently (time-only compression), we predict the position of each vertex v of frame f from three of its neighbors in frame f and from the positions of v and of these neighbors in the previous frame (space-time compression). We introduce here two extrapolating spacetime predictors: the ELP extension of the Lorenzo predictor, developed originally for compressing regularly sampled 4D data sets, and the Replica predictor. ELP may be computed using only additions and subtractions of points and is a perfect predictor for portions of the animation undergoing pure translations. The Replica predictor is slightly more expensive to compute, but is a perfect predictor for arbitrary combinations of translations, rotations, and uniform scaling. For the typical 3D animations that we have compressed, the corrections between the actual and predicted value of the vertex coordinates may be compressed using entropy coding down to an average ranging between 1.37 and 2.91 bits, when the quantization used ranges between 7 and 13 bits. In comparison, space-only compression yields a range of 1.90 to 7.19 bits per coordinate and time-only compressions yields a range of 1.77 to 6.91 bits per coordinate. The implementation of the Dynapack compression and decompression is trivial and extremely fast. It perform a sweep through the animation, only accessing two consecutive frames at a time. Therefore, it is particularly well suited for realtime and out-of-core compression, and for streaming decompression. In this paper, we propose a new framework to perform motion compression for time-dependent 3D geometric data. Temporal coherence in dynamic geometric models can be used to achieve significant compression, thereby leading to efficient storage and transmission of large volumes of 3D data. The displacement of the vertices in the geometric models is computed using the iterative closest point (ICP) algorithm. This forms the core of our motion prediction technique and is used to estimate the transformation between two successive 3D data sets. The motion between frames is coded in terms of a few affine parameters with some added residues. Our motion segmentation approach separates the vertices into two groups. Within the first group, motion can be encoded with a few affine parameters without the need of residues. In the second group, the vertices need further encoding of residual errors. Also in this group, for those vertices associated with large residual errors under affine mapping, we encode their motion effectively using Newtonian motion estimates. This automatic segmentation enables our algorithm to he very effective in compressing time-dependent geometric data. Dynamic range data captured from the real world, as well as complex animations created using commercial tools, can be compressed efficiently using this scheme. Compression of 3D mesh animations is a topic that has received increased attention in recent years, due to increasing capabilities of modern processing and displaying hardware. In this paper we present an improved approach based on known techniques, such as principal component analysis (PCA) and EdgeBreaker, which allows efficient encoding of highly detailed dynamic meshes, exploiting both spatial and temporal coherence. We present the results of our method compared with similar approaches described in literature, showing that using our approach we can achieve better performance in terms of rate distortion ratio. Time-varying mesh, which is attracting a lot of attention as a new multimedia representation method, is a sequence of 3-D models that are composed of vertices, edges, and some attribute components such as color. Among these components, vertices require large storage space. In conventional 2-D video compression algorithms, motion compensation (MC) using a block matching algorithm is frequently employed to reduce temporal redundancy between consecutive frames. However, there has been no such technology for 3-D time-varying mesh so far. Therefore, in this paper, we have developed an extended block matching algorithm (EBMA) to reduce the temporal redundancy of the geometry information in the time-varying mesh by extending the idea of the 2-D block matching algorithm to 3-D space. In our EBMA, a cubic block is used as a matching unit. MC in the 3-D space is achieved efficiently by matching the mean normal vectors calculated from partial surfaces in cubic blocks, which our experiments showed to be a suboptimal matching criterion. After MC, residuals are transformed by the discrete cosine transform, uniformly quantized, and then encoded. The extracted motion vectors are also entropy coded after differential pulse code modulation. As a result of our experiments, 10 -18 compression has been achieved. In this paper, we present a representation for three-dimensional geometric animation sequences. Different from standard key-frame techniques, this approach is based on the determination of principal animation components and decouples the animation from the underlying geometry. The new representation supports progressive animation compression with spatial, as well as temporal, level-of-detail and high compression ratios. The distinction of animation and geometry allows for mapping animations onto other objects. 3D meshes are commonly used to represent virtual surface and volumes. However, their raw data representations take a large amount of space. Hence, 3D mesh compression has been an active research topic since the mid 1990s. In 2005, two very good review articles describing the pioneering works were published. Yet, new technologies have emerged since then. In this article, we summarize the early works and put the focus on these novel approaches. We classify and describe the algorithms, evaluate their performance, and provide synthetic comparisons. We also outline the emerging trends for future research. We describe a compression method for three-dimensional animation sequences that has notable advantages over existing techniques. We first aggregate the frame data by similarity and reorganize them into clusters, which results in the sequence split into several motion fragments of varying lengths. To minimize the number of clusters and obtain optimal clustering, we perform frame alignment, which eliminates the “global” rigid transformation from each frame data and use only “pose” when evaluating the similarity between frames. We then apply principal component analysis for each cluster, from which we get coordinates of corresponding frames in a reduced dimension. Because similar frames are considered, the number of coefficients required for each frame becomes smaller; thus, we obtain better dimension reduction for a given reconstruction error. Further, we perform intracluster compression based on linear coding. Because every motion fragment presents similar frames, conventional linear predictive coding can be replaced by key frame-based linear coding to achieve minimal reconstruction error. Results show that our method can obtain a high compression ratio, with a limited reconstruction error. Copyright © 2013 John Wiley & Sons, Ltd.