COAST: COntrollable Arbitrary-Sampling NeTwork for Compressive Sensing

Recent deep network-based compressive sensing (CS) methods have achieved great success. However, most of them regard different sampling matrices as different independent tasks and need to train a specific model for each target sampling matrix. Such practices give rise to inefficiency in computing and suffer from poor generalization ability. In this paper, we propose a novel COntrollable Arbitrary-Sampling neTwork, dubbed COAST, to solve CS problems of arbitrary-sampling matrices (including unseen sampling matrices) with one single model. Under the optimization-inspired deep unfolding framework, our COAST exhibits good interpretability. In COAST, a random projection augmentation (RPA) strategy is proposed to promote the training diversity in the sampling space to enable arbitrary sampling, and a controllable proximal mapping module (CPMM) and a plug-and-play deblocking (PnP-D) strategy are further developed to dynamically modulate the network features and effectively eliminate the blocking artifacts, respectively. Extensive experiments on widely used benchmark datasets demonstrate that our proposed COAST is not only able to handle arbitrary sampling matrices with one single model but also to achieve state-of-the-art performance with fast speed. The source code is available on https://github.com/jianzhangcs/COAST.


I. INTRODUCTION
As a typical inverse problem, compressive sensing (CS) aims to recover an unknown natural signal from a small number of its measurements acquired by a linear random projection, which has been successfully applied in single-pixel imaging [1], [2], accelerating magnetic resonance imaging (MRI) [3], sparse-view computed tomography (CT) [4], wireless telemonitoring [5] and cognitive radio communication [6].
Mathematically, supposing that x ∈ R N is the original natural signal and Φ ∈ R M ×N is a linear random projection (sampling matrix), the CS measurement of x, denoted by y ∈ R M , is usually formulated as where n denotes the additive white Gaussian noise with standard deviation σ. By default, σ = 0 and y is called 'noiseless'. The purpose of CS is to infer x from its randomized CS measurement y. Because M N , this inverse problem is typically ill-posed, whereby CS ratio, denoted by γ, is defined as γ = M N . This paper mainly focuses on CS reconstruction of natural images, where x ∈ R N is a vectorized representation of an image patch of size CS reconstruction results from the seen and unseen sampling matrices (γ = 10%) produced by a recent state-of-the-art method, i.e., ISTA-Net + [9], and our proposed COAST. It is clear to observe that COAST significantly surpasses the ISTA-Net + , especially for the unseen sampling matrix.
proposed framework can be easily extended to various types of data, e.g., MRI and video data [7], [8].
In the past decade, a great number of image CS reconstruction methods, which include traditional model-driven methods and data-driven deep neural methods, have been developed. The traditional model-driven methods [10], [11] commonly convert the reconstruction problem to a Maximum a Posteriori (MAP) estimation problem and adopt an iterative manner to search for the optimal solution that can fit both the given measurement and prior constraints. Although these modeldriven methods enjoy the advantage of interpretability, due to the iterative nature of the solutions and the hand-crafted characteristics, they suffer from high computational complexity and are difficult to characterize various complicated image structures. Fueled by the rise of deep learning, several datadriven deep neural methods [12], [13], [14], [15], [16], [17], [18], [19] have been recently proposed for image CS reconstruction by direct learning the inverse mapping from the CS measurement domain to the original signal domain. Compared ISTA-Net + (Unseen) Fig. 2. PSNR values of ISTA-Net + and our COAST on BSD68 dataset with respect to various training epochs in the case of γ = 10%. It can be observed that for the seen sampling matrix, both ISTA-Net + and COAST can be improved by the training process. For the unseen sampling matrix, the reconstruction performance of ISTA-Net + decreases with the increase of training epochs, while our COAST still achieves very similar performance with the seen sampling matrix.
to the model-driven methods, the data-driven deep neural methods dramatically reduce time complexity due to their non-iterative nature of the solutions and achieve impressive reconstruction performance. However, they are often trained as a black box, with limited insights from the CS domain.
Although existing deep network-based methods have achieved impressive results, most of them regard different sampling matrices as different independent tasks and need to train a specific model for each target sampling matrix with fixed CS ratio and patch size. For the convenience of discussion, we consider these methods as Φ-specific methods, due to the fact that they need to be re-trained for various sampling matrices. In Fig. 1, we give an example and make a comparison between a state-of-the-art CS model, i.e., ISTA-Net + [9] and our COAST. As we can see, ISTA-Net + can produce a desirable result for the specific target (seen) sampling matrix, but can not handle other (unseen) sampling matrix. As shown in the green and gray bars of Fig. 2, we can see that ISTA-Net + tend to overfit to the specific data used for training. Specifically, with the increasing of the epoch number, its PSNR performance increases on the seen sampling matrix but decreases on the unseen sampling matrix. In brief, we can learn that current deep network-based models trained on a specific sampling matrix suffer from poor generalization ability and can not handle arbitrary sampling matrices well with a single model. For different sampling matrices, they usually need to re-train different network models separately. Obviously, such practices result in large storage space and high time complexity, and do not satisfy the needs of the real scenarios that usually contain various sampling matrices.
In this paper, to address the above issues, we propose a novel COntrollable Arbitrary-Sampling neTwork, dubbed COAST, for compressive sensing. As a Φ-agnostic method, our COAST is only trained once and can handle arbitrary sampling matrices (including unseen sampling matrices) with one single model. In particular, COAST consists of a sampling subnet (SS), an initialization subnet (IS), and a recovery subnet (RS). A random projection augmentation (RPA) strategy in SS is proposed as the core component to handle arbitrary sampling problem with one single model. Moreover, a controllable proximal mapping module (CPMM) and a plug-and-play deblocking (PnP-D) strategy are proposed to improve the robustness and enhance the performance of COAST. Extensive experiments fully verify the effectiveness of our COAST. Specifically, for the arbitrary seen sampling matrices, our COAST is able to achieve competitive results with the corresponding single models, which are re-trained for different sampling matrices. For the arbitrary unseen sampling matrices, COAST can achieve very similar performance with the corresponding seen sampling matrices (see the Gray bar in Fig. 2). More encouragingly, as shown in Fig. 2, COAST significantly outperforms the previous state-of-the-art ISTA-Net + [9], especially for the unseen sampling matrices.
Overall, the main contributions of this paper are as follows: • We propose a novel COntrollable Arbitrary-Sampling neTwork (COAST) for CS, which is able to handle arbitrary sampling with one single model. • A random projection augmentation (RPA) strategy is developed to promote the generalization ability and the performance of networks, which can also be directly incorporated into existing deep network-based CS methods. • A controllable proximal mapping module (CPMM) and a plug-and-play deblocking (PnP-D) strategy are further proposed to further improve the robustness and enhance the reconstruction quality. • Our COAST achieves state-of-the-art performance for arbitrary sampling matrices with one single model, promoting the application in real-world CS systems.

II. BACKGROUND
This section will present some recent related works, and briefly introduce traditional iterative shrinkage-thresholding algorithm (ISTA) [25] for facilitating the follow-up discussion.

A. Related works
We generally group existing compressive sensing (CS) reconstruction methods of images into two categories: traditional optimization-based CS methods and recent network-based CS methods. In what follows, we give a brief review of both and focus on some recent work most relevant to our own.
Optimization-based Methods: Given the input linear measurements y = Φ(x + n), traditional image CS methods usually reconstruct the original imagex by solving the following optimization problem: wherex and Φ denote the recovered image patch and linear random projection (sampling matrix), respectively, and ψ(x) denotes the hand-crafted image prior-regularized term with λ being the (generally pre-defined) regularization parameter. Optimization-based Methods are based on regularization terms inspired by image priors. In the early stage, the prior knowledge about transform coefficients (e.g. statistical dependencies [26], [27], sparsity [28], etc.) are exploited to reconstruct visual images. Furthermore, some elaborate priors exploiting the non-local self-similarity properties of natural images have been proposed to improve CS reconstruction [29], [30], [31], [32], [33]. Recently, some fast and effective convolutional neural network (CNN) denoisers are trained and integrated into Half Quadratic Splitting (HQS) [34] and alternating direction method of multipliers (ADMM) [35], [36] to solve image inverse problems. However, all these traditional image CS methods require hundreds of iterations by means of various iterative solvers (e.g. ISTA [25], ADMM [37], or AMP [38]), which inevitably increase computational cost thus restricting the application of CS. In addition, the selected image prior (e.g. optimal transform) or the optimization parameters (e.g. step size and regularization parameter) are usually handcrafted and quite challenging to pre-define.
Network-based Methods: Recently, inspired by the powerful learning capability of deep networks and its success in computer vision tasks, some network-based methods have been proposed. [39] first proposes to apply a stacked denoising auto-encoder (SDA) to learn the representation from training data and to reconstruct test data from the CS measurements. [12] and [40] separately propose to use fully-connected neural networks for image and video CS reconstruction. [41] further develops a CNN-based CS method, dubbed ReconNet, which learns to regress an image patch (output) from its CS measurement (input). In order to erase the blocking artifacts and further enhance the performance, [14] proposes a samplingrecovery framework called CSNet. Moreover, a joint sampling and scalable reconstruction CS network is proposed, dubbed SCSNet, allowing the one well-trained model to deal with several carefully-designed sampling matrices [17]. However, SCSNet can not be generalized to arbitrary sampling matrices.
Lately, some deep unfolding networks, e.g., ADMM-Net [7], ISTA-Net [9], DPDNN [42], and OPINE-Net [43], are proposed to combine the merits of both optimization-based and network-based methods. Specifically, [7] proposes a so-called ADMM-Net architecture by reformulating ADMM for CS magnetic resonance imaging (CS-MRI) using deep networks. ISTA-Net [9], which works well for both general CS and CS-MRI based on ISTA, goes beyond that to adopt nonlinear transforms to more effectively sparsify natural images and develops an efficient strategy for solving their proximal mapping problems. DPDNN [42] is designed to solve common inverse problems by unfolding the denoising process inspired by HQS method. Another deep unfolding model dubbed OPINE-Net [43] maps an optimization problem into the deep network for joint adaptive binary sampling and recovery of image CS for the first time. In addition, based on the idea that the structure of the auto-encoder network itself is a good prior to capture image statistics, an unsupervised method DIP [44] is proposed to directly fit the network for each corrupted image. To further improve the performance of DIP and overcome its dependence on the number of iterations, NLR-CSNet [45] includes nonlocal prior in addition to deep network prior and constructs an HQS-based optimization method for network learning.
Weakness: As we can see, recent research on CS has achieved great success due to the rapid development of deep convolutional neural networks. However, the CS problem of arbitrary sampling matrices has been ignored for a long time. Most existing works regard different sampling matrices as different independent tasks. To handle arbitrary sampling matrices, a) exiting traditional optimization-based methods usually need to adopt an iterative manner to re-search for the optimal solution that can fit each specific sampling matrix; and b) most of network-based methods need to re-train a network model separately for each sampling matrix, otherwise it will suffer from poor performance on the new given sampling matrix (see Fig. 1). Such re-search and re-train practices inevitably result in both time-consuming and storage-consuming issues.

B. Traditional ISTA for CS
The iterative shrinkage-thresholding algorithm (ISTA) is a popular first-order proximal method, which is well suited for solving many large-scale linear inverse problems. Specifically, ISTA solves the CS reconstruction problem in Eq. (2) by iterating between the following update steps: x (k) = arg min where k is the ISTA iteration index and ρ is the step size. Eq. (3) is trivial, while Eq. (4) is actually a special case of the so-called proximal mapping, i.e., prox λψ (r (k) ). Mathematically, the proximal mapping of regularizer ψ denoted by prox λψ (r) is defined as: prox λψ (r) = arg min How to handle prox λψ (r) in an efficient and effective way is of great importance for ISTA.

III. APPROACH
In this section, we elaborate on the design of our proposed COAST. We first formulate our COAST in Section III-A. Then, as illustrated in Fig. 3, our COAST consists of a sampling subnet (SS), an initialization subnet (IS) and a recovery subnet (RS), which are described in Section III-B. Finally, we describe the training details in Section III-C.

A. Problem Formulation
In CS of natural images, the common practice is to partition an image into several non-overlapped image patches and sample each patch independently with the same sampling matrix. Assuming we have an original image patch x and a sampling matrix Φ, we get the CS measurement y = Φ(x + n), where n denotes the additive white Gaussian noise with standard deviation σ. In order to improve reconstruction performance and increase network capacity, instead of hand-crafted, we make the image prior-regularized term ψ(·) as learnable parameters, and adopt a residual neural network to solve its proximal mapping operator effectively. Therefore, we obtain the following general problem formulation: wherex denotes the recovered image patch.

B. Architecture Design of COAST
To map the optimization in Eq. (6) into deep network efficiently, we propose to implement it in a three-step scheme. Concretely, the first step is to simulate the sampling process and promote the training diversity. Then, the second step is to deal with the dimensionality mismatch between the original image and its CS measurement. The final step is to design the recovery network to generate the reconstructed image.
Accordingly, as illustrated in Fig. 3, our COAST consists of three subnets: a sampling subnet (SS), an initialization subnet (IS) and a recovery subnet (RS).
1) Sampling Subnet (SS): As illustrated in Fig. 3, we incorporate the proposed random projection augmentation (RPA) strategy into the sampling subnet (SS) to boost the generalization ability and the reconstruction performance. Here, we describe the SS at testing and training stages, respectively.
Testing Stage: RPA is not used at testing stage of SS. Similar to previous works [41], [9], in SS we just employ the sampling matrix Φ ∈ R M ×N to acquire the randomized Training Stage: Similar with the popular data augmentation techniques [46], our RPA on sampling matrices is only adopted at the training stage of SS. In the following, we will describe the training stage of SS and detail the RPA on how and why it works.
In previous works, as shown in Fig. 4(a), Φ 1 ∈ R M1×N1 is a fixed sampling matrix used for training and testing, which means that the resulting network model is Φ-specific. Fig. 4(b), the proposed RPA augments the training by introducing additional different sampling matrices. For a given sampling matrix with dimension M -by-N , denoted by Φ M ×N , we introduce the RPA operator by adopting similar ways to generate its augmented set, including N S sampling matrices with the same dimension as Φ M ×N . Thus, the proposed RPA operator is defined as:

For our COAST, as shown in
Furthermore, the RPA operator is extended to deal with a set of multiple sampling matrices with L(L ≥ 2) different arbitrary dimensions, i.e., arbitrary CS ratios and patch sizes, which is defined as follows: Accordingly, the sampling matrices augmented by RPA form a set {Φ t } NΦ t=1 , where the total number of the augmented sampling matrices is N Φ = L × N S . It is worth emphasizing that our RPA can be applied to two widely-used types of sampling matrices, i.e. fixed random Gaussian matrix (FRGM) and data-driven adaptively learned matrix (DALM), which are respectively elaborated as follows: • For FRGM, each sampling matrix Φ is constructed by generating a random Gaussian matrix and then orthogonalizing its rows, i.e. ΦΦ = I, where I is the identity matrix. In this case, we implement our RPA by generating multiple FRGMs using different random seeds multiple times for argumentation. • For DALM, each sampling matrix Φ is regarded as a learnable network parameter and is adaptively learned by jointly optimizing the sampling matrix and the reconstruction operator with training dataset. In practical implementation, we implement the proposed RPA by utilizing OPINE-Net [43] multiple times to generate multiple DALMs for argumentation. Now, we introduce how to incorporate the proposed RPA into our CS model. With RPA, we extend the fixed sampling matrix Φ 1 (see Fig. 4(a)) to its augmented set {Φ t } NΦ t=1 (see Mt, Nt represent the dimensions of each sampling matrix. As we can see, the previous approaches only accept one sampling matrix to learn Φ-specific models, while our RPA augments the sampling matrices by introducing additional sampling matrices to enable our COAST to handle arbitrary sampling matrices with one single model. Fig. 4(b)) and acquire the CS measurement y t = Φ t (x + n). So we can obtain the following optimization framework with the proposed RPA by replacing the fixed Φ in Eq. (6) with {Φ t } NΦ t=1 , which is formulated as: In the practical implementation of casting Eq. (9) into deep network, we randomly select a sampling matrix from {Φ t } NΦ t=1 for each batch. Accordingly, in the following, the subscript t is omitted without confusion for simplicity. Compared with Eq. (6), Eq. (9) makes the CS model Φagnostic. On one hand, RPA augments the training and alleviates the over-fitting problem. On the other hand, the learning from multiple augmented data by RPA will significantly improve the generalization capability of network. Besides, RPA can enable our COAST to jointly learn different knowledge from different sampling matrices which further promotes the performance of each sampling matrix. In brief, the RPA enjoys two advantages: promoting the generalization capability of CS models and promoting the performance of each sampling matrix. The effectiveness of RPA will be validated in the experimental section.
2) Initialization Subnet (IS): Similar to traditional ISTA, our COAST also requires an initialization, which is denoted byx (0) in Fig. 3. In particular, to conduct the initialization process, previous network-based methods, such as [12], [41], usually exploit the parameterized fully-connected layer, due  5. Illustration of the proposed controllable proximal mapping block (CPMB), which introduces a controllable unit (CU) into a standard residual block. Specifically, the CU, which implemented by a fully-connected layer, takes the condition vector z as input to generate the controllable vector, which is used to dynamically modulate the output features of the residual block.
to the dimensionality mismatch between the original image and its CS measurement. However, such practice inevitably gives rise to a high amount of parameters and restricts the CS models to only accept the input measurements generated by the specific sampling matrix with the specific fixed dimension. In order to enable our CS model to handle arbitrary sampling matrices without introducing new parameters, in this paper, we propose to initially set zero values tox (0) and set ρ (0) to be 1, i.e.x (0) = 0. Interestingly, this is equivalent to set x (0) = Φ y. The reason is that both the initializations ofx (0) and ρ (0) will lead to the same r (1) , i.e. r (1) = Φ y, according to Eq. (10). Intuitively, the design of our IS is a naive solution to handle various sampling matrices with arbitrary dimension.
3) Recovery Subnet (RS): Considering the simplicity and interpretability, we design a deep unfolding model named Recovery Subnet (RS) to solve Eq. (6). In this paper, we map traditional ISTA into deep network to construct our RS as [9]. It is worth noting that as a deep unfolding model, the RS structurally enables our COAST to handle arbitrary sampling matrices with one single model. Specifically, compared with classical deep network methods, deep unfolding methods enable the sampling matrix to play a more important role, which gives more potential of boosting the generalization ability on arbitrary sampling matrix.
As illustrated in Fig. 3, the recovery subnet consists of N P phases and each phase is composed of a gradient descent module (GDM) and a controllable proximal mapping module (CPMM), which are corresponding to the two update steps in traditional ISTA, i.e. Eq. (3) and Eq. (4). Besides, at the front and back ends of CPMM, we incorporate the proposed plugand-play deblocking (PnP-D) strategy at the testing stage.
Gradient Descent Module (GDM): To preserve the ISTA structure, gradient descent module (GDM) is directly defined according to Eq. (3), where the step size ρ becomes a learnable parameter. Takingx (k−1) , Φ, y as input, the output of GDM in k-th phase, denoted by H (k) GDM , is finally defined as: Controllable Proximal Mapping Module (CPMM): Different from ISTA-Net + [9], we propose a more effective and robust CPMM to solve the proximal mapping problem in Eq. (5) with learnable ψ(·). In addition to r (k) from GDM, CPMM also takes a condition vector z = [γ, σ] as input, where γ is CS ratio and σ is the standard deviation of additive white Gaussian noise in Eq. (1). In this paper, γ ∈ {10%, 20%, 30%, 40%, 50%} and σ ∈ [0, 1]. That is to say, CPMM can dynamically modulate the network features according to the given γ and σ. Concretely, as shown in Fig. 3, CPMM in the k-th phase, denoted as H (k) CPMM , is composed of N C controllable proximal mapping blocks (CPMBs), two convolution layers, and a long skip connection, which can be formulated as follows: CPMB,j . In our implementation, we set N C = 3 by default. Due to its simplicity and effectiveness, the standard residual block (RB) is adopted as the building block for CPMB, which consists of two convolution layers and a ReLU activation layer. In particular, as illustrated in Fig. 5, each CPMB is composed of a controllable unit (CU) and a residual block (RB), wherein the output of CU is used to modulate the output of the last convolution layer in RB. Each CU, denoted by H CU is efficiently implemented by a fully-connected layer, and takes the condition vector z as input to generate a controllable vector H CU (z) with dimension C, which can be defined as: where W CU is a fully-connected layer. Furthermore, in our preliminary experiments, it is found that the parameter sharing of W CU across all the CUs does not impair the performance. Therefore, we utilized one W CU for our COAST to reduce the number of parameters. With H CU (z), the process of CPMB can be formulated as: where F j−1 ))). In our experimental implementation, we set C = 32 by default.
Plug-and-Play Deblocking (PnP-D): To address the issue of blocking artifacts, we propose a plug-and-play deblocking (PnP-D) strategy at testing stage, which is composed of two parts: plug-and-play folding (PnP-F) and plug-and-play unfolding (PnP-U). To be concrete, each image is partitioned into N B non-overlapped image patches. As illustrated in Fig. 6(a), without PnP-D, each patch is independently processed by the network and its network feature with dimension N B ×C × √ N × √ N also goes through CPMM independently. However, with PnP-D, in each phase, our COAST first uses PnP-F to combine N B sliding local patches into a whole image's feature with dimension Then, the fully-convolutional CPMM operates on this folded image's feature by exploiting the inter-block relationship in feature domain to effectively eliminate the blocking artifacts. From the output of CPMM, PnP-U is to further extract N B sliding local patches with dimension N B × C × √ N × √ N . Obviously, PnP-D is only exploited at the testing stage and enables our COAST to sample independently and reconstruct jointly without introducing any extra parameters. Thus, PnP-D can be used as a plugin and easily embedded into a wide range of existing CS systems without changing the original model structures. Experiments will demonstrate the effectiveness and generalization capability of our PnP-D strategy.

C. Network Parameters and Loss Function
Given L(L ≥ 1) sampling matrices with different dimensions M l by N l (l ≤ L), we first utilize RPA to generate the augmented sampling matrix set and taking y i,t , Φ t , z i as input, COAST aims to reduce the discrepancy between x i and H COAST (y i,t , Φ t , z i ). Therefore, we design the end-to-end loss function for our COAST as follows: where N D denotes the total number of training patches of size And Θ denotes the learnable parameter set in COAST, includes the parameters of the GDM and CPMM H

A. Implementation Details
Following common practice [41], [9] and for fair comparison, we employ the same set of 91 images provided in [48] in the training stage and extract the luminance component of 88912 randomly cropped image patches (each of default size 33 × 33), i.e., N D = 88912 and N = 1089.
Our COAST is implemented in PyTorch [49] and is trained on a workstation with Intel Core i7-6820 CPU and GTX1080Ti GPU with a batch size 64. We use Adam [50], as well as momentum of 0.9 and weight decay of 0.999 for parameter optimization. Training COAST with phase number N P = 20 roughly takes four days. For testing, we utilize two widelyused benchmark datasets: Set11 [41] and BSD68 [51]. Note that we deal with color images in the transformed YCbCr space and the CS recovered results are evaluated with Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) [52] on Y channel (i.e., luminance).

B. Comparison with State-of-the-Art Methods
Our proposed COAST is compared with ten representative state-of-the-art methods including TVAL3 [22], BM3D-AMP [38], LDAMP [22], DIP [44], ReconNet [16], DPDNN [42], GDN [47], ISTA-Net + [9], NLR-CSNet [45], and OPINE-Net [43]. Here, the acquired measurement is 'noiseless', i.e., the standard deviation σ of n in Eq. (1) is set to 0. To be concrete, TVAL3, BM3D-AMP are traditional optimizationbased methods; ReconNet is a classical deep network-based method; GDN, LDAMP, DPDNN, ISTA-Net + , and OPINE-Net are deep unfolding methods; DIP and NLR-CSNet are unsupervised methods based on the idea that the structure of the auto-encoder network itself is a good prior to capture image statistics. Specifically, when handling multiple sampling matrices, ReconNet, GDN, LDAMP, DPDNN, and ISTA-Net + need to re-train the model with respect to each sampling matrix, while TVAL3, BM3D-AMP, DIP, and NLR-CSNet need to solve the computationally expensive optimization with hand-crafted prior or deep image prior for each image. However, our COAST can handle arbitrary sampling matrices with a single trained model, leading to high performance and fast speed. According to whether a network model needs to be re-trained or not for various sampling matrices, we generally divide CS methods into two categories: Φ-specific and Φ-agnostic. In the following, three comparisons with different sampling matrices are provided, i.e., fixed random Gaussian matrix (FRGM), data-driven adaptively learned matrix (DALM) and unseen sampling matrix (USM).
Comparison on FRGM: For FRGM, to demonstrate the effectiveness of our approach, we utilize the same five specific sampling matrices as [9], which are separately denoted by t=1 to train our COAST only once. Table I lists the average PSNR and SSIM results for different ratios on Set11 and BSD68 datasets, respectively. One can observe that, as a deep unfolding method, ISTA-Net + has a great improvement compared with most of the     Table I show the run-time analysis of all the competing methods, which clearly indicates that the proposed COAST produce consistently better reconstruction results, while remaining computationally attractive real-time speed. Moreover, among all the Φ-agnostic methods, our COAST has the fastest running speed with GPU. Fig. 7 and Fig. 8 further show the visual comparisons of all the competing methods on two test images when CS ratio γ is 10% and 30% respectively. Obviously, the proposed COAST is able to recover more texture details and much sharper edges than other competing methods.
Comparison on DALM: For DALM, three learned sampling matrices produced by OPINE-Net [43] are used, which are separately denoted by,   Table II and Table I, one can observe that the same method with the same CS ratio in the case DALM generally obtains more than 1 dB gains than that in    FRGM, which indicates that DALM can preserve more image information than FRGM. Moreover, Table II clearly shows that our COAST not only achieves the best performance among all the competing methods, but also enables dealing with multiple DALMs with a single model, which is in accordance with the results on FRGM. In particular, our COAST improves roughly 0.32 dB and 0.21 dB on average PSNR over three CS ratios, in comparison with the state-of-the-art method OPINE-Net on Set11 and BSD68 datasets, respectively. Fig. 9 and Fig. 10 illustrate the visual results of various methods and further demonstrate the effectiveness and superiority of our method.
Comparison on USM: To verify the generalization ca-pability of our COAST, we further use the RPA operator under the condition of FRGM to generate several unseen sampling matrices with γ ∈ {10%, 20%, 30%, 40%, 50%}, i.e., t=1 . Without additional training, we directly employ the learned models in Table I for these unseen sampling matrices and show the PSNR and SSIM results of Set11 dataset in Table III for various CS ratios. Undoubtedly, the Φ-specific ISTA-Net + [9], ReconNet [16] and DPDNN [42] can not be generalized to arbitrary unseen sampling matrices and perform very poorly, while our COAST outperforms the computationally expensive TVAL3 and BM3D-AMP and still yields impressive performance on par with the sampling matrix in {Φ t } 125 t=1 , exhibiting good generalization capability. Fig. 11 and Fig. 12 provide the visual comparison in the case of USM for several methods on two test images in BSD68 when CS ratio is 30% and 50%, respectively, which clearly shows that our COAST is not affected and produces best results. Moreover, Table IV reports the average PSNR results for four other unseen sampling matrices with γ / ∈ {10%, 20%, 30%, 40%, 50%}, showing similar performance to the model separately trained for each one, which again demonstrates the effectiveness and generality of our COAST to deal with arbitrary sampling matrices.

C. Single Model for Arbitrary Sampling Matrices
In order to verify the effectiveness and robustness of our COAST to handle arbitrary sampling matrices with one single model, we further carry out CS reconstruction experiments from noiseless and noisy CS measurements in the cases of various CS ratios, patch sizes, and noise levels for FRGM. Reconstruction from noiseless CS measurements: Define a set of CS ratios as Γ = {10%, 20%, 30%, 40%, 50%} and a set of patch sizes as P = {32 2 , 33 2 , 36 2 , 40 2 , 48 2 }. Then, we generate a set of FRGMs, denoted by Ω = {Φ Mi×Ni }, where Mi Ni ∈ Γ and N i ∈ P, 1 ≤ i ≤ 25. Obviously, Ω has 25 different sampling matrices with various dimensions. Here, the COAST model trained by the augmented set of Ω, i.e., RPA(Ω) (including 25 × N S sampling matrices) is referred to as COAST-A, while each COAST model separately trained by the augmented set of each Φ Mi×Ni , i.e., RPA(Φ Mi×Ni ) (including N S sampling matrices) is referred to as COAST-S. N S is set to be 25 and σ is set to be 0.
The PSNR results between COAST-A and COAST-S in the cases of various CS ratios and patch sizes on the Set11 dataset are reported in Table V. One can clearly observe that COAST-A with a single model is able to handle various arbitrary CS ratios and arbitrary patch sizes effectively, and achieves CS performance on par with COAST-S, with PSNR distances being all below 0.2 dB. Surprisingly, COAST-A can even achieve better performance than the corresponding single model COAST-S in the case of γ = 10% and √ N = 33. Table V fully verifies the effectiveness and generalization of our COAST to deal with arbitrary sampling matrices.
Reconstruction from noisy CS measurements: To further demonstrate the robustness of our COAST to noise, we conduct CS reconstruction experiments from noisy CS measurements, i.e., the noise deviation σ of the additive white Gaussian measurement noise n in Eq. (1) is not set to be zero. Define a set of CS ratios as Γ = {10%, 20%, 30%, 40%, 50%}. At training, we adopt the same five sampling matrices in Table I, which constitute a set, dubbed The augmented set of Ω, i.e., RPA(Ω) (including 5 × N S sampling matrices) with σ = 0, are used to train our model, which is referred to as COAST-C. N S is set to be 25. In addition, the COAST model trained by RPA(Ω) and σ ∈ [0, 10] (i.e., σ is randomly uniform-sampled from [0, 10]) is referred to as COAST-N. Similar to the COAST-S in the previous noiseless experiment, we separately employ the augmented set of each Φ Mi×N , i.e., RPA(Φ Mi×N ) (including N S sampling matrices) with σ = 5/10 to train our model and denote it by COAST-S.
The PSNR results between COAST-S and COAST-N in the cases of different noise levels and CS ratios on the Set11 dataset are reported in Table VI. As we can see, COAST-N with one single model is quite robust to noise and achieves almost the same performance as the corresponding separately trained model COAST-S, with their PSNR distance being less than 0.1dB for all the tasks. More encouragingly, in the case of γ = 10% and σ = 5, COAST-N even surpasses COAST-S. In Fig. 13, we compare the performance between COAST-N and COAST-C in the cases of various noise levels and CS ratios. One can clearly observe that COAST-N achieves better results than COAST-C as the noise level increases, which further verifies the robustness and effectiveness of our COAST.

D. Ablation Studies and Discussions
This subsection will present the ablation study to investigate the contribution of each component in our proposed COAST on the default settings of FRGM.
Effect of RPA: As a core component in COAST, random projection augmentation (RPA) strategy augments the training in the sampling space and enjoys two main advantages: promoting performance and promoting generalization. The experiments in Subsection IV-B on unseen sampling matrix (USM) have fully verified the generalization ability of our  COAST. Here, we mainly evaluate the advantage of promoting performance. We first implement COAST to conduct experiments on two types of sampling matrices, i.e., fixed random Gaussian matrix (FRGM) and data-driven adaptively learned matrix (DALM) 1 . Table VIII reports the average PSNR results of our COAST trained with various values of N S in the case of γ = 30% for FRGM and DALM. Note that N S = 1 means no additional sampling matrix is introduced and the RPA strategy is not used. From Table VIII Table VII, we observe that RPA greatly promotes CS performance across all the cases. Effect of CU in CPMM: The controllable proximal mapping module (CPMM) is able to dynamically modulate the network features and greatly promote the network robustness by adopting the controllable unit (CU). The experiments in Subsection IV-C on reconstruction from noisy CS measurements have substantially proved that CU has the advantage of high noise robustness. Here, we mainly emphasize that the utilization of CU can also bring about performance improvement. Settings (b) and (c) in Table VII provide the performance comparison between CPMM with CU and CPMM without CU for five CS ratios on Set11. One can observe that CPMM with CU consistently outperforms CPMM without CU across all the ratios and obtains on average 0.07 dB gains.
Shared vs. Unshared: In our experiment, the recovery subnet (RS) of our COAST has 20 phases and each phase has 3 CUs. Thus, there exist 60 CUs in total in COAST. Here, we study the effect of the parameter sharing across all the CUs. Note that each CU has 2×32+32 = 96 parameters. Therefore, the shared strategy successfully reduces the parameter number from 96 × 60 = 5760 to 96. Settings (c) and (d) in Table VII provide the PSNR comparison between unshared strategy and shared strategy. Obviously, shared strategy achieves almost the same performance as unshared strategy, which fully demonstrates the effectiveness of our shared strategy, greatly reducing the parameter number without performance loss.
With vs. Without PnP-D: The plug-and-play deblocking (PnP-D) strategy can be easily integrated into existing CS frameworks. Settings (d) and (e) in Table VII give the PSNR comparison between w/ PnP-D and w/o PnP-D. It is clear to see that PnP-D strategy greatly boosts the performance across all ratios, with the most significant improvement up to 0.93 dB, which fully verifies the effectiveness of PnP-D. Fig. 14 further shows the visual results by w/ PnP-D and w/o PnP-D, which clearly demonstrates that the PnP-D is able to eliminate the blocking artifact effectively, resulting in highquality reconstructed images.
Performance of Phase Number: Since each phase of COAST corresponds to one iteration in traditional ISTA, it is expected that larger phase number N P will lead to higher reconstruction performance. Fig. 15 shows the average PSNR curve by COAST for Set11 with respect to different phase numbers in the case of γ = 10%. One can observe that the PSNR curve increases as N P increases. However, the curve is becoming almost flat when N P ≥ 20. Thus, considering the trade-off between computational complexity and reconstruction performance, we set the default value of phase number N P to be 20 in COAST.
Discussion: As far as we know, there exist three methods with the ability of handling arbitrary CS ratios, i.e., SCSNet [17], RACS [53] and DRNN [54]. We compare them with our COAST in the following paragraphs.
Although SCSNet [17] and RACS [53] can also deal with arbitrary sampling ratios using only one model, there exist two main differences between their methods and our COAST: scope of application w.r.t sampling matrices and mechanism of handling arbitrary CS ratios. Firstly, our COAST is generally applicable whenever a sampling matrix is given beforehand (including FRGM and DALM), while SCSNet and RACS learn the sampling matrix and are not applicable when one wants to use any given sampling matrix, whether it is learned or not. Secondly, the mechanisms of how to enable the network to handle arbitrary CS ratios are different. Specifically, SCSNet achieves sampling and reconstruction at any sampling ratio by using a greedy method to select the measurement bases. RACS trains a single-ratio reconstruction network first, and then 'adapts' new sampling matrices to the parameter-fixed reconstruction network. For our COAST, the proposed RPA enables our reconstruction network to jointly learn different knowledge from multiple sampling matrices, thus enabling our COAST to robustly deal with arbitrary CS ratios.
As for DRNN [54], two main aspects differentiate DRNN and our COAST: application field and object of operation. Firstly, DRNN aims to perform classification directly on compressive measurements, while our COAST aims to recover the unknown natural signal from its compressed measurements. Secondly, the training scheme of DRNN can be considered as a ratio-level operation, and ignores the matrix-level. Different from DRNN, our proposed RPA is not limited to ratio-level, but further promotes the performance of each sampling matrix by adopting the matrix-level argumentation, which is clearly verified in Table VIII in our paper.

E. Analysis on USM for Deep Unfolding Methods
To analyze why the unfolding based methods (e.g., ISTA-Net + ) fail with unseen sampling matrices (USM), we discuss the dependence of the deep unfolding networks on the specific sampling matrix in the initialization process and the reconstruction process, respectively.
Initialization: To deal with the dimensionality mismatch between the original image and its CS measurements, deep network-based methods require an initialization. Some methods [9] depend on the specific sampling matrix used in training, while some methods [22] don't (e.g., directly initialized by 0). ISTA-Net + , which belongs to the former category, first uses the training dataset and the specific training sampling matrix to compute a linear mapping matrix Q init , and then obtainsx (0) =Q init y. Due to the fixed dimension of Q init in initialization, ISTA-Net + can not handle sampling matrices in unseen CS ratios. When handling unseen sampling matrices in seen CS ratio, ISTA-Net + still performs very poorly, as shown in Table III.
Based on the above analysis, one may wonder whether deep unfolding methods would still fail with USM when removing the dependence on the sampling matrix in initialization process.
Reconstruction: To eliminate the effect of initialization and focus on how the reconstruction process influences the generalization ability on USM, we initially set zero values in ISTA-Net + and compare the average PSNR results for ISTA-Net + [9], BM3D-AMP [38] and COAST on unseen sampling matrices across different CS ratios on Set11 dataset (shown in Table IX). The number in the grey box indicates the PSNR gain (↑) or drop (↓) compared with corresponding seen sampling matrices, which represents how general the method is when being extended to unseen sampling matrices. We can observe that, for the traditional iterative method BM3D-AMP, the PSNR performance fluctuation shown in grey boxes is mild, indicating a nice generality w.r.t unseen sampling matrices. However, when applying the deep unfolding network ISTA-Net + to unseen sampling matrices, the PSNR performance drops a lot (see the grey boxes for ISTA-Net + ), especially on unseen ratios (underlined in Table IX,), which demonstrates that ISTA-Net + exhibits inferior generality of being extended to unseen sampling matrices. As a comparison, the PSNR performance gain or drop is quite slight for our COAST, thus verifying its superior generality of handling USM. One might think that, ISTA-Net + is a special case of traditional ISTA when testing, and should be robust to arbitrary sampling matrices just like traditional iterative methods. But this is in fact not the case, as compared earlier. Considering the differences between traditional iterative methods and deep unfolding models, we attribute such failure to two aspects.
Firstly, the selected image prior (e.g., optimal transform) and the optimization parameters (e.g., step size and regularization parameter) are usually hand-crafted in traditional iterative methods without considering any specific sampling matrix, while in deep unfolding models, all the parameters (e.g., optimal transforms, step sizes, shrinkage thresholds, etc.) are learned from training data pairs, which contain the information of a specific training sampling matrix. Therefore, these learned parameters are somewhat dependent on the training sampling matrix, and thus leading to inferior generality when being applied to unseen sampling matrices.
Secondly, although traditional optimization-based methods and deep unfolding models are both iterative, the iteration number is much smaller for the latter. Taking ISTA-Net + as an example, it is forced to converge after 9 iterations, thus leading to relatively larger step sizes and stronger proximal mapping, which indicates fewer and larger iteration steps towards the optimal solution when optimizing the target problem. However, when being applied to unseen sampling matrices, the target problem may change a lot, and therefore, larger iteration steps might relatively lead to a suboptimal result, thus resulting in inferior generality w.r.t. unseen sampling matrices.
To sum up, for deep unfolding models, the above two reasons cause the network parameters to 'overfit' the specific sampling matrix used in training, and thus leading to inferior generality when handling USM.

V. CONCLUSION
In this paper, a novel COntrollable Arbitrary-Sampling neTwork, dubbed COAST, is proposed for compressive sensing under the optimization-inspired deep unfolding framework. COAST is able to deal with arbitrary sampling matrices with one single model, exhibiting good interpretability, high robustness as well as attractive computational speed. Extensive experiments demonstrate that COAST not only achieves CS performance on par with single models separately trained for different specific sampling matrices, but also greatly improves current state-of-the-art CS results on both fixed random Gaussian matrix (FRGM) and data-driven adaptively learned matrix (DALM). It is expected that the proposed COAST will inspire new insights for a wide range of potential realworld applications in computational imaging in the future, including but not limited to handling arbitrary sampling masks in accelerating MRI, dealing with multiple digital micromirror devices (DMDs) in single-pixel imaging and reconstructing under different system matrices in sparse-view CT. Future work includes the direct extensions of COAST on MRI data and video data applications.