Limited angle CT reconstruction is an under-determined linear inverse problem that requires appropriate regularization techniques to be solved. In this work we study how pre-trained generative adversarial networks (GANs) can be used to clean noisy, highly artifact laden reconstructions from conventional techniques, by effectively projecting onto the inferred image manifold. In particular, we use a robust version of the popularly used GAN prior for inverse problems, based on a recent technique called corruption mimicking, that significantly improves the reconstruction quality. The proposed approach operates in the image space directly, as a result of which it does not need to be trained or require access to the measurement model, is scanner agnostic, and can work over a wide range of sensing scenarios. Computed Tomography (CT) reconstruction is the process of recovering the structure and density of objects from a series of x-ray projections, called sinograms. While traditional full-view CT is relatively easier to solve, the problem becomes under-determined in two crucial scenarios often encountered in practice -(a ) few-view: when the number of available x-ray projections is very small, and ( b) limited-angle: when the total angular range is less than 180 degrees, as a result of which most of the object of interest is invisible to the scanner. These scenarios arise in applications which require the control of x-ray dosage to human subjects, limiting the cost by using fewer sensors, or handling structural limitations that restrict how an object can be scanned. When such constraints are not extreme, suitable regularization schemes can help produce artifact-free reconstructions. While the design of such regularization schemes are typically driven by priors from the application domain, they are found to be insufficient in practice under both few-view and limited-angle settings. In the recent years, there is a surge in research interest to utilize deep learning approaches for challenging inverse problems, including CT reconstruction [1, 2, 3] . These networks implicitly learn to model the manifold of CT images, hence resulting in higher fidelity reconstruction, when compared to traditional methods such as Filtered Backprojection (FBP), or Regularized Least Squares (RLS), for the same number of measurements. While these continue to open new opportunities in CT reconstruction, they rely of directly inferring mappings between sinograms and the corresponding CT images, in lieu of regularized optimization strategies. However, the statistics of sinogram data can vary significantly across different scanner types, thus rendering reconstruction networks trained on one scanner ineffective for others. Furthermore, in practice, the access to the sinogram data for a scanner could be restricted in the first place. This naturally calls for entirely image-domain methods that do not require access to the underlying measurements. In this work, we focus on the limited-angle scenario, which is known to be very challenging due to missing information. Instead of requiring sinograms or scanner-specific representations, we pursue an alternate solution that is able to directly work in the image domain, with no pairwise (sinogram-image) training necessary. To this end, we advocate the use of generative adversarial networks (GANs) [4] as image manifold priors. GANs have emerged as a powerful, unsupervised technique to parameterize high dimensional image distributions, allowing us to sample from these spaces to produce very realistic looking images. We train the GAN to capture the space of all possible reconstructions using a training set of clean CT images. Next, we use an initial seed reconstruction using an existing technique such as Filtered Back Projection (FBP) or Regularized Least Squares (RLS) and 'clean' it by projecting it onto the image manifold, which we refer to as the GAN prior following [6] . Since the final reconstruction is always forced to be from the manifold, it is expected to be artifact-free. More specifically, this process involves sampling from the latent space of the GAN, in order to find an image that resembles the seed image. Though this has been conventionally carried out using projected gradient descent (PGD) [5, 6 ], as we demonstrate in our results, this approach performs poorly when the initial estimate is too noisy or has too many artifacts, which is common under extremely limited angle scenarios. Instead, our approach utilizes a recently proposed technique referred to as corruption mimicking, used in the design of MimicGAN [7] , that achieves robustness to the noisy seed reconstruction through the use of a randomly initialized shallow convolutional neural network (CNN), in addition to PGD. By modeling the initial guess of this network as a random corruption for the unknown clean image, the process of corruption mimicking alternates between estimating the unknown corruption and finding the clean solution, and this alternating optimization is repeated until convergence, in terms of effectively matching the observed noisy data. The resulting algorithm is test time only, and can operate in an artifact-agnostic manner, i.e. it can clean images that arise from a large class of distortions like those obtained from various limited-angle reconstructions. Furthermore, it reduces to the well-known PGD style of projection, when the CNN is replaced by an identity function. In figures 1, 2, we show qualitative and quantitative results obtained for both the MNIST and Fashion-MNIST datasets respectively. In both cases, we demonstrate significant improvements in recovering the true reconstruction compared to the vanilla GAN prior. It should be noted that a performance boost of nearly 4-5 dB on MNIST and 0.5-1dB on Fashion-MNIST are achieved with no additional information or data, but due to the inclusion of the robust GAN prior. Additionally, PSNR and SSIM tend to be uncorrelated with perceptual metrics in many cases, as perceptually poor reconstructions can be deceptively close in PSNR or SSIM. A potential fix in GAN-based reconstruction approaches is to compute error in the discriminator feature space as a proxy for perceptual quality. [8] : Given the RLS reconstruction, we improve them by projecting onto the image manifold using corruption mimicking [7] . In all cases, we show the improvement obtained by using the robust GAN prior over a standard GAN projection.
We show that robust GAN priors work better than GAN priors for limited angle CT reconstruction which is a highly under-determined inverse problem.
{"src": "scitldr-AIC_train", "paper_id": "B1gznX3q8S"}