We propose a new joint image reconstruction method by recovering edge directly from observed data. More specifically, we reformulate joint image reconstruction with vectorial total-variation regularization as an @math minimization problem of the Jacobian of the underlying multi-modality or multi-contrast images. Derivation of data fidelity for Jacobian and transformation of noise distribution are also detailed. The new minimization problem yields an optimal @math convergence rate, where @math is the iteration number, and the per-iteration cost is low thanks to the close-form matrix-valued shrinkage. We conducted numerical tests on a number multi-contrast magnetic resonance image (MRI) datasets, which show that the proposed method significantly improves reconstruction efficiency and accuracy compared to the state-of-the-arts.
Besides joint TV or tensor based regularization, the parallelism of level sets across multi-modality contrast images are also proposed as joint image regularization in @cite_3 @cite_12 @cite_17 @cite_13 . The main idea is to exploit the structural similarity of two images @math and @math measured by the parallelism of their gradients @math and @math at each point using @math , for some functions @math and @math . Then the regularization in joint reconstruction takes form @math Several different choices of @math and @math have been studied in these works. For instance, in @cite_5 , @math and @math are taken as identities, or @math and @math . In @cite_13 , @math is the identity and @math . In @cite_12 the side information on the level set, namely, the location and direction of the gradients of a reference image, is available to assist the reconstruction.
@cite_3 @cite_5 @cite_13 @cite_12 @cite_17
2134884567 2885775891 1960698473 2016802777
Clinical imaging with structural MRI routinely relies on multiple acquisitions of the same region of interest under several different contrast preparations. This work presents a reconstruction algorithm based on Bayesian compressed sensing to jointly reconstruct a set of images from undersampled k-space data with higher fidelity than when the images are reconstructed either individually or jointly by a previously proposed algorithm, M-FOCUSS. The joint inference problem is formulated in a hierarchical Bayesian setting, wherein solving each of the inverse problems corresponds to finding the parameters (here, image gradient coefficients) associated with each of the images. The variance of image gradients across contrasts for a single volumetric spatial position is a single hyperparameter. All of the images from the same anatomical region, but with different contrast properties, contribute to the estimation of the hyperparameters, and once they are found, the k-space data belonging to each image are used independently to infer the image gradients. Thus, commonality of image spatial structure across contrasts is exploited without the problematic assumption of correlation across contrasts. Examples demonstrate improved reconstruction quality (up to a factor of 4 in root-mean-square error) compared to previous compressed sensing algorithms and show the benefit of joint inversion under a hierarchical Bayesian model. We present an approach named JSFusion (Joint Sequence Fusion) that can measure semantic similarity between any pairs of multimodal sequence data (e.g. a video clip and a language sentence). Our multimodal matching network consists of two key components. First, the Joint Semantic Tensor composes a dense pairwise representation of two sequence data into a 3D tensor. Then, the Convolutional Hierarchical Decoder computes their similarity score by discovering hidden hierarchical matches between the two sequence modalities. Both modules leverage hierarchical attention mechanisms that learn to promote well-matched representation patterns while prune out misaligned ones in a bottom-up manner. Although the JSFusion is a universal model to be applicable to any multimodal sequence data, this work focuses on video-language tasks including multimodal retrieval and video QA. We evaluate the JSFusion model in three retrieval and VQA tasks in LSMDC, for which our model achieves the best performance reported so far. We also perform multiple-choice and movie retrieval tasks for the MSR-VTT dataset, on which our approach outperforms many state-of-the-art methods. Even after two decades, the total variation (TV) remains one of the most popular regularizations for image processing problems and has sparked a tremendous amount of research, particularly on moving from scalar to vector-valued functions. In this paper, we consider the gradient of a color image as a three-dimensional matrix or tensor with dimensions corresponding to the spatial extent, the intensity differences between neighboring pixels, and the spectral channels. The smoothness of this tensor is then measured by taking different norms along the different dimensions. Depending on the types of these norms, one obtains very different properties of the regularization, leading to novel models for color images. We call this class of regularizations collaborative total variation (CTV). On the theoretical side, we characterize the dual norm, the subdifferential, and the proximal mapping of the proposed regularizers. We further prove, with the help of the generalized concept of singular vectors, that an $ ^ ... Currently sparse signal reconstruction gains considerable interest and is applied in many fields. In this paper, a similarity induced by joint sparse representation is designed to construct the likelihood function of particle filter tracker so that the color visual spectrum and thermal spectrum images can be fused for object tracking. The proposed fusion scheme performs joint sparse representation calculation on both modalities and the resultant tracking results are fused using min operation on the sparse representation coefficients. In addition, a co-learning approach is proposed to update the reference templates of both modality and enhance the tracking robustness. The proposed fusion scheme outperforms state-of-the-art approaches, and its effectiveness is verified using OTCBVS database.